{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " \n",
    "<br>\n",
    "<h1 align=\"center\">《关于中文文本情感分析的实验报告》</h1>\n",
    " \n",
    "<br><br>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 问题：中文文本情感分析\n",
    "\n",
    "查阅相关资料，对上述提供的约12万条微博中文文本数据集，完成下述情感分析问题。\n",
    "\n",
    "通过中文文本数据预处理（如删除非中文字符等）；使用jieba库分词（pip install jieba)；引入停用词 （stop_words.txt)；特征编码（词频或tf-idf或其它编码方法)， 将文本数据转化为特征向量。然后将正负向评论两类数据集都分别按7：3比例分为训练集和测试集。使用合适的分类算法，分别给出在训练集和测试集上的 `准确率，精度，召回率 和F1得分`指标，并画出ROC曲线，给出曲线下方面积值。\n",
    "\n",
    "要求：提交一份jupyter notebook 格式的文档,文档名用本队成员姓名命名；该文档主要包含如下内容\n",
    "\n",
    "1. 对中文自然语言处理这一研究方向做简单介绍；\n",
    "2. 提供三种完整解决方案（包括改进方案），并从训练时间，测试时间，测试得分等方面比较三种解决方案的优缺点。 每种方案都应该有相应的算法原理介绍，算法步骤，代码实现，实验结果及分析等\n",
    "3. 评估及结论 \n",
    "4. 相应的参考文献  \n",
    "5. 在文档后面应附上每位队成员所做的主要工作（如是算法原理介绍，代码编写，实验分析，改进方案，分类模型优缺点分析等）\n",
    "\n",
    "作业提交时间: 2022.6.20前每队负责人将作业发到邮箱， 并附上队成员信息（学号，姓名）。\n",
    "### "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 中文自然语言处理简介"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 自然语言处理的概览\n",
    "&emsp;&emsp;自然语言处理的研究从二战结束后的机器翻译和人工智能的研究算起的话，己经有长达半个多世纪的发展历程。自然语言的处理所涉及的任务主要有: `机器翻译、信息检索、文本概括、人机问答、信息提取、主题模型`，现在对语法、句法的研究越来越突出。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 自然语言的三个发展阶段   \n",
    "\n",
    "+   **早期自然语言处理**\n",
    "\n",
    "&emsp;&emsp;第一阶段(60~80年代):基于规则来建立词汇、句法语义`分析、问答、聊天和机器翻译系统`。好处是规则可以利用人类的内省知识，不依赖数据，可以快速起步；问题是覆盖面不足，像个玩具系统，规则管理和可扩展一直没有解决。 \n",
    "+   **统计自然语言处理**\n",
    "\n",
    "&emsp;&emsp;第二阶段(90年代开始)：基于统计的机器学习(ML)开始流行，很多NLP开始用`基于统计的方法`来做。主要思路是利用带标注的数据，基于人工定义的特征建立机器学习系统，并利用数据经过学习确定机器学习系统的参数。运行时利用这些学习得到的参数，对`输入数据进行解码，得到输出`。机器翻译、搜索引擎都是利用统计方法获得了成功。\n",
    "+  **神经网络自然语言处理** `[1]`\n",
    "\n",
    "&emsp;&emsp;第三阶段(2008年之后)：深度学习开始在语音和图像发挥威力。随之，NLP研究者开始把目光转向`深度学习`。先是把深度学习用于`特征计算`或者建立一个新的特征，然后在原有的统计学习框架下体验效果。比如，搜索引擎加入了深度学习的检索词和文档的相似度计算，以提升搜索的相关度。自2014年以来，人们尝试直接通过`深度学习建模`，进行`端对端的训练`。目前已在机器翻译、问答、阅读理解等领域取得了进展，出现了深度学习的热潮。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 中文自然语言处理\n",
    "\n",
    "&emsp;&emsp;对于中文自然语言处理，相对于英语，又有所不同，于中文自然语言处理，相比于英语等有天然分隔符的语言，难度更大。例如，界定单词`边界的词法分析`任务，也被称为“中文自动分词”的任务，是中文语言处理的首要步骤。其中，`新词发现、歧义切分`等关键问题又是`基于对语言的理解`之的。中文自然语言处理是以转换自然语言为基本原则，遵循基于规则、基于统计等研究思路，并且适当添加一些可靠的实例来完成翻译过程。\n",
    "\n",
    "&emsp;&emsp;中文自然语言处理主要体现在`中文自然语言信息主题知识获取瓶颈`处理上，因为中文自然语言处理的领域以及时间的处理、对机器翻译的知识性等都应当进一步结合这种模式的深度和难度，最后完成中文自然语言处理的`传统信息知识处理`。需要注意的是应当根据语义网来更加直观的分析系统规划的各个流程，然后依据适应性来适当增加对中文自然语言有关知识的系统支持和关注。`[2]`\n",
    "\n",
    "&emsp;&emsp;随着人工智能相关技术的不断发展，中文自然语言处理和计算机的结合问题也将会面临大量的知识成本，这就要求从中文自然语言的处理、发展历程以及传统处理手段等来进一步对建模的核心问题完成相应的处理。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 问题分析"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 整体思路   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> &emsp;&emsp;这是典型的 `中文文本情感分析` 问题，对此我们通常先进行 `中文自然语言` 处理。  \n",
    "> 然后选取合适的向量化方法进行`特征编码`，选取合适的分类模型进行`分类训练`。\n",
    "> \n",
    "> &emsp;&emsp;观察到问题所提供的数据规模较大(12万条)，对此我们提出以下优化方向：\n",
    "> - 使用因特尔提供的 `sklearnex 加速`方案，减少训练耗时。\n",
    "> - 评估出合适的`特征编码`方案，减少编码过程中的内存占用。\n",
    "> - 使用`网格搜索`，通过不同参数组合尝试去获取最优的模型。\n",
    "> - 相对批量学习，使用`在线学习`进行增量学习，减少内存占用。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 逐题分析"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 1. **对中文文本数据进行预处理，保存到新文件**  \n",
    "> - 对 12 万条样本数据逐条进行以下处理：去除非中文字符、过滤单网名、分词处理、去除停用词。\n",
    "> - 由于样本数据规模较大，我们建议将预处理好的数据保存到新文件中，避免重复做无用功。\n",
    "\n",
    "> 2. **特征编码，将文本数据转化为特征向量**  \n",
    "> - CountVectorizer 需要在内存保存全部的词汇，无法调用其做核心学习。\n",
    "> - TfidfVectorizer 需要把训练集的所有特征向量保存在内存，以计算逆文档频率。\n",
    "> - HashingVectorizer 无需在内存中存储词汇表，占用内存非常低，可应用于大型数据集。\n",
    "\n",
    "> 3. **将正负向评论按 7:3 比例分为训练集和测试集**\n",
    "> - 针对`批量学习`算法，我们将调用 `sklearn` 的库函数进行划分。\n",
    "> - 针对`在线学习`算法，每次拟训练1000条样本数据，12万条可划分为120批，\n",
    "> - 前 85 批用来进行小批次增量学习，余下35000条用来进行测试。\n",
    "\n",
    "> 4. **提供三种方案，给出训练集和测试集上的 `准确率，精度，召回率 和F1得分`**\n",
    "> - 逻辑回归模型(Logistic)\n",
    "> - 随机梯度下降模型(SGD)\n",
    "> - 基于在线学习的 SGD\n",
    "\n",
    "> 5. **画出ROC曲线，给出曲线下方面积**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 数据预处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 中文文本数据预处理\n",
    "\n",
    "- 原样本为前 6 万条为正类，后 6 万条为负类，避免在线学习出现过拟合现象，需对样本数据进行 `洗牌处理`。\n",
    "- 去除样本中的网名昵称、非中文字符，再对中文文本 `分词处理，去除停用词`，去除空语句。\n",
    "- 将预处理完成的数据写入 weibo_senti.csv，以供减少模型训练所需时间。\n",
    "\n",
    "- `preprocess()`：预处理样本数据，并将处理好的数据保存到新文件中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Intel(R) Extension for Scikit-learn* enabled (https://github.com/intel/scikit-learn-intelex)\n"
     ]
    }
   ],
   "source": [
    "from sklearnex import patch_sklearn\n",
    "from sklearn.utils import shuffle\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import warnings\n",
    "import jieba\n",
    "import csv\n",
    "import re\n",
    "\n",
    "patch_sklearn() # 用以加速模型训练\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "pd_all = pd.read_csv('weibo_senti_100k.csv')    # 加载原始样本数据\n",
    "pd_shuffled = shuffle(pd_all, random_state=100) # 打乱原始样本数据\n",
    "file_processed = \"weibo_senti.csv\"      # 用以命名已预处理数据的文件\n",
    "\n",
    "# 读入停用词列表，用于匹配删除文本中的停用词\n",
    "stop_words = [line.strip() for line in open(\"stop_words.txt\", 'r').readlines()]\n",
    "\n",
    "def preprocess(origidata=pd_shuffled, newfile=file_processed, stopwords=stop_words):\n",
    "    \"\"\" 对12万条样本(origidata)中的评论进行预处理，保存到 `newfile` 中，以供模型训练。\n",
    "        \n",
    "        读取原始样本数据，针对原始样本的每条评论进行分词处理，以空格分隔词语，\n",
    "        去除网名、停用词等，仅保留中文字符，最后以(label,review)的形式存入文件。\n",
    "\n",
    "        例如，正类评论保存为：1, 更博 爆照 帅 越来越 爱 生快 傻 缺爱 爱 爱。\n",
    "    \"\"\"\n",
    "    with open(newfile, 'w', encoding='UTF8', newline='') as f:\n",
    "        writer = csv.writer(f)\n",
    "        writer.writerow(['label', 'review'])\n",
    "\n",
    "        for _, row in origidata.iterrows():\n",
    "            # 对单条语句进行文本预处理，过滤网名，去除非中文字符。\n",
    "            seten = re.sub('@.*?[\\s:：]|[^\\u4e00-\\u9fa5]', '', row['review'])\n",
    "\n",
    "            # 利用 jieba 库对 单条语句 进行分词处理\n",
    "            seten = jieba.lcut(seten)\n",
    "\n",
    "            # 如果语句中的词语不是停用词(即不再停词列表中)，则加入列表\n",
    "            seten = [word for word in seten if word not in stopwords]\n",
    "\n",
    "            if seten: # 如果语句不为空，则写入文件，即清洗掉空语句\n",
    "                # 将(标签，分词过的单条语句)存入CSV文件，以供后续使用\n",
    "                writer.writerow([row['label'], ' '.join(seten)])\n",
    "                "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 批量学习依赖函数\n",
    "\n",
    "- `tokenizer()`：对已预处理的数据直接令牌化，推荐使用。\n",
    "- `tokenizer_process`：具有预处理功能的令牌化函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def tokenizer(text: str)-> list:\n",
    "    \"\"\" 针对 `已利用preprocess()预处理` 的数据进行 `令牌化` 。  \n",
    "\n",
    "        已预处理的12万条评论，其中每条评论已经分词处理，词语间以空格分隔，\n",
    "        去除网名、停用词，仅含中文字符，因此不需要再去除停用词。\n",
    "        例如，正类评论：“更博 爆照 帅 越来越 爱 生快 傻 缺爱 爱 爱”。\n",
    "        只需要将上述样本转成列表，传入模型训练即可。\n",
    "    \"\"\"\n",
    "    return text.split()\n",
    "\n",
    "def tokenizer_process(text: str)-> list:\n",
    "    \"\"\" 针对未预处理的数据进行令牌化\n",
    "        需要进行中文字符过滤处理、分词处理等\n",
    "        为加快模型训练，建议使用 `函数tokenizer()`\n",
    "    \"\"\"\n",
    "    seten = re.sub('@.*?[:：]|[^\\u4e00-\\u9fa5]', '', text)\n",
    "    seten = jieba.lcut(seten)\n",
    "\n",
    "    return seten"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 在线学习依赖函数\n",
    "\n",
    "- `stream_docs()`：生成器函数，每次小批量读入数据并返回一个文档，以供在线学习使用。\n",
    "- `get_minibatch()`：每次从生成器中获取小批次数据，可用于获取训练集、测试集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def stream_docs(path):\n",
    "    \"\"\" 迭代读取CSV文件，返回(标签, 评论) \"\"\"\n",
    "    try:\n",
    "        with open(path, 'r', encoding='utf-8') as csv:\n",
    "            next(csv)  # 跳过第一行的标题\n",
    "            for line in csv:\n",
    "                label, text = int(line[0]), line[2:]\n",
    "                yield label, text\n",
    "    except:\n",
    "        # 若找不到已预处理的CSV文件，则重新预处理\n",
    "        print(\"找不到文件路径，请调用函数 preprocess() 预处理数据\")\n",
    "        # preprocess()\n",
    "\n",
    "def get_minibatch(doc_stream, size):\n",
    "    \"\"\" 从生成器中返回指定条数的数据 \"\"\"\n",
    "    docs, y = [], []\n",
    "    try:\n",
    "        for _ in range(size):\n",
    "            label, text = next(doc_stream)\n",
    "            docs.append(text)\n",
    "            y.append(label)\n",
    "    except StopIteration:\n",
    "        return docs, y\n",
    "    return docs, y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 模型训练\n",
    "\n",
    "- 在此节中，我们选取以下三种方案进行分类训练:  \n",
    "- 1.基于网格搜索的逻辑回归模型\n",
    "- 2.基于网格搜索的随机梯度下降\n",
    "- 3.基于在线学习算法的梯度下降"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 基于 `逻辑回归` 算法分类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.1.1 **logistic** 回归算法理论\n",
    "\n",
    "&emsp;&emsp;logistic 回归模型学习时，对于给定的训练数据集 \n",
    "$T = \\lbrace (x_1,y_1),(x_2,y_2),...,(x_N,u_N) \\rbrace $ ,其中， $x_i \\in R^n,y_i \\in {0,1}$ ,\n",
    "\n",
    "&emsp;&emsp;可以应用 `极大似然估计法` 估计模型参数，从而得到 logistic 回归模型。\n",
    "\n",
    "&emsp;&emsp;设： $$P(Y=1|x)= \\pi(x),\\quad P(Y=0|x)=1- \\pi(x)$$ \n",
    "\n",
    "&emsp;&emsp;似然函数为: $$ \\prod_{i=1}^{N} [\\pi(x_i)]^{y_i}[1-\\pi(x_i)]^{1-y_i} $$\n",
    "\n",
    "&emsp;&emsp;对数似然函数为$$\\begin{align} L(w) &=\\sum_{i=1}^{N}[y_i log\\pi(x_i)+(1-y_i)log(1-\\pi(x_i))]\\\\ &=\\sum_{i=1}^{N}[y_i log\\frac{\\pi(x_i)}{1-\\pi(x_i)}+log(1-\\pi(x_i))]\\\\ &=\\sum_{i=1}^N [y_i(w \\cdot x_i)-log(1+e^{(w \\cdot x_i)})] \\end{align}$$\n",
    "\n",
    "&emsp;&emsp;对 $L(w)$ 求极大值 ，得到 $w$ 的估计值。\n",
    "\n",
    "&emsp;&emsp;假设 $w$ 的极大似然估计值是 $\\hat(w)$ ,那么得到的 `logistic 回归模型`为: $$P(Y=1|x)=\\frac{e^{(\\hat{w}\\cdot x)}}{1+e^{(\\hat{w}\\cdot x)}} \\\\ P(Y=0|x)=\\frac{1}{1+e^{(\\hat{w}\\cdot x)}}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.1.2 训练集和测试集的划分\n",
    "\n",
    "- 读取预处理后的CSV文件，数据为乱序的标签及评论对，一共12万条。\n",
    "- 将12万条已预处理的数据按照 7:3 划分为训练集和测试集。\n",
    "- 划分后的训练集测试集可供 4.1节的逻辑回归模型，4.2节的SGD模型训练。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 加载 12 万条已预处理好的样本数据集\n",
    "# 若找不到已预处理的文件，则调用函数预处理\n",
    "# preprocess() \n",
    "df_processed = pd.read_csv(file_processed)\n",
    "X_all = df_processed['review']\n",
    "y_all = df_processed['label']\n",
    "\n",
    "# 按照 7:3 分隔训练集、测试集\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X_all, y_all,test_size=0.3, random_state=100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.1.3 基于 **`网格搜索`** 的 **`logistic`** 回归模型训练\n",
    "- 利用网格搜索尝试寻找最优模型。\n",
    "- 为评估三种模型的训练速度等各项指标，网格搜索均已使用模型各自的最优参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn.linear_model import LogisticRegression, SGDClassifier\n",
    "\n",
    "\n",
    "pipe = Pipeline([\n",
    "    (\"tfid\", TfidfVectorizer()),\n",
    "    (\"logit\", LogisticRegression())\n",
    "])\n",
    "\n",
    "param_grid = [{\n",
    "    \"tfid__ngram_range\": [(1, 1)],\n",
    "    \"tfid__tokenizer\": [tokenizer],\n",
    "    \"logit__C\": [1],\n",
    "    \"logit__solver\": [\"sag\"]\n",
    "}]\n",
    "\n",
    "logis = GridSearchCV(pipe, param_grid, n_jobs=-1)\n",
    "\n",
    "logispara = logis.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.1.4 **logistic** 回归模型的各项评分及 ROC\n",
    "- 函数 **`estimate_model()`** 输出以下指标：\n",
    "- 训练集及测试集上的准确率、精度、召回率、F1得分。\n",
    "- 训练集及测试集上的ROC曲线、ROC曲线下的面积"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练集上的准确率：0.95380,\t测试集上的准确率：0.93280\n",
      "训练集上的精度：  0.93908,\t测试集上的精度：0.91465\n",
      "训练集上的召回率：0.97039,\t测试集上的召回率：0.95428\n",
      "训练集上的F1得分：0.95448,\t测试集上的F1得分：0.93404\n",
      "训练集上ROC面积: 0.98624, 测试集上ROC面积: 0.97764\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAHgCAYAAAC1uFRDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABLWElEQVR4nO3dd5xkZZn3/89VoXP3hJ6cCYOACALDAGIgSU7+QJQV2HFdWQMLPBgeH0FB1lXXtLsYF11XUBHBFRyQsKCCujNEyUMa4iQm9KQO013hXL8/zumemp4O1TNdVXOmvu/Xq19V59SpOlefnulv3+fc577N3REREZH4SVS6ABEREdkxCnEREZGYUoiLiIjElEJcREQkphTiIiIiMaUQFxERialUpQsYqQkTJvicOXMqXYaIiEhZPPbYY+vcfeJAr8UuxOfMmcOjjz5a6TJERETKwsxeH+w1nU4XERGJKYW4iIhITCnERUREYkohLiIiElMKcRERkZhSiIuIiMSUQlxERCSmFOIiIiIxpRAXERGJKYW4iIhITCnERUREYkohLiIiElMKcRERkZhSiIuIiMRUyULczH5iZmvM7JlBXjczu9bMlprZU2Z2SKlqERER2R2VsiX+U+CkIV4/GZgbfV0E/KCEtYiIiOx2Shbi7v4nYP0Qm5wJ3OChB4GxZja1VPWIiIiUQpAPyGSybOnqon3TejatX1u2fafKtqftTQeWFSwvj9atqkw5IiKlFQRO4E7gELiH6wqWPQBn63LgjkfPc3nHAyefzxDkspDvIcj14NluCPLk83mCfJ5MNkOKAA/y4HkSmXYCErg77gEEAR4E4XMPH90dD/J4sHVduOM8iSADnocgAM9jngcPsOjz8YCGLW/Sk24Bdxwv+Bxwz4fr3aN14WMQBIBj4UY4jnm4Dg9IeZZkkMU8j+HhowfhF+FnJshj7iQ9R9Jz0VH2rZ+LY3i4qvd59JoVLrP1dSvYNnydvueG08omurwWgCQBSfKkLKCm4Oe8iUa4emVp/zFFKhniRTOziwhPuTNr1qwKVyOy6/AoAPJ94eDR862BkQ+cfEEYuPcLDt/6OdsFR+AEwdbHfN/nh6GyobOHhpoEnsvhuS0E+SxBPgf5HLlcDs/nsCBLKtdJ4GCeC3/BBzksyEGQpz7bRsbqoqDI49Ev6t6A6A0NgoBkkCXhuegXevjVk8mSsjw15qS895d+GCJ9v7D7wsExeoPAyWRzJBOQsnC7KcEq2mx89Ms8iLaN3oOT6F3X97qTJstk2mhjTBQWvb/0Awy2C4NEv/DYGhCQICAB0TZbX0vgJMlTY/kK/CsrXtaT0XeWwCH844HCSCz4ssLvvN/RMiNHkh6ri45kArdE9HmGW5LAEuSj9W615BIpIEH4MeFRg/A5GBY99r1mbLtN7+sUPDeLXo9+Ima8itOQb2dz3VRIpMASkEhhiWS4nKzBaxo5okzHvJIhvgKYWbA8I1q3HXe/DrgOYN68eV760qRa5AOnO5unO5snm3ey+YCeXEBHT47ubJ617T2kk4m+ICwMtGwQsLa9h6baFD25gEwufG8+CLYNvyjwlm3oYmJzXfh6PgzDXOB09OTY2JWhuS7dty6XD1i1sZOxNZC23tDLkvIeEkEOghwe5Eh6PmoNBKTJUWNZ0uRIkydFnjR5aixLHRlqyTDFNpCMQmhvW8FmGqghT5ocKXLURtumyFMXfUbK8sywdbR7PQAp8iQISJEnaZX/75gjSd4TBJYkIEHW0gW/7Atjs9+yJcCMTB6SySSQgGSKuflXWZueHm5niegXfgK3KGatdzkMlbwZK4MZGM6W9FjCX/6JgiAASGAGbgnMElgUYIXBEn4lws2jbcDwhGGE7wtStXgiDcnoK5EmSNZhyRRYEksmyeSNutoaSCSxRBKzJAnyBDXN4b4TiWh9WGMiWsYSJJMJzJKYWd+2JMNgSvR+XiIJiSSJZBhgfeuTaSxhGJBOJkiYkU4YCSP6XqQUKhniC4GLzewm4HBgk7vrVHqVyOUDunMBWzJhgPbk8mzakiNwJ5ML6M7m6crkyeQCckFANh8GW08uoCuTpzuXZ0smT082IFvwem+QZvIB2XzAsvVbaKlP4Q7ZaF0uHwV3LiAfhO2EJAFNbKGRbppsC2lyfeGYJE8SJ2Hhcoo8DXRTQ4605aghxwxbS87raUjkqU0E1JCj1nLUWyYMScsx2dsIludIp1PUkGV6sJIua4xaeHlSHfkoIPOkPUsykYfcAAcvwU73ZsmmGgkStaSzm2lv2RtPpAkStXiikSBZR5CogUQaT6YgkWZ1IkE6u5lM08woHMKWR2AJ0uk0lkiR6tlIMGYmiWQaSyZJJlNYMk0imSKR7YTmKeH7kmkSySSWSIctFw+gpjEMpSggw9ZNMnqeLHiegHR9tC4FiQQpRv8XWcsof55IqZQsxM3sl8DRwAQzWw5cBaQB3P2HwJ3AKcBSoAv4cKlqkdGRywds6MqyvjNDR0+Wto4MAD1R6K5p72FdRw+phNGdDejM5HhpdQeppJHJBby8toOm2jQdPVm6s0GRe3Ua6GG8tTOZ9dRbhpm2BkskaU7mmJTMMM46qbcMewTL6Ei2UGN5UgSkLU8qlce2dNOUzJH2LCmyTMivoDvVSCIVkPRswbW0UZIIg49kDaRqIVW3teXUtRFa50JtEyT3pbFzHUzYO3pP9L7e03Lp+r7WVniaLh1+fpCHxgnR9smC9yYhWRvus3f7RDpsSaUbIV0HqfAz0wUtozGj+92LSBmVLMTd/bxhXnfgk6XavwzO3VnfmaGtM8OazT20dfbQ3p1jQ2eG9V0Z2rtzbNqSZW17Dxu6MmzekqUzahUXa1xDmvp0kvENKbq3dHDQ+Bzv2qOLVPd6Dk6voD6RZUywgTrvZlzny+RrxpAKemjc/DJB3VgS+R5qut4cfkd5wqCqbQpDs/slGL9nGFyJFCRqIdkE6YYoUGvBEtTlumHMzIKgS4etwdrmcNu+1l5vK7DfY6oWaqJ9JmvC96cbwucJjaEkIuURi45tUrx84Kze3M2a9h6Wre9i9eZu3tzUzctrO1jfmWFDV5bVm7vpGSSQm2pTjKlP01yXYkJTLbPGN9Bcl6KpLkVDOsX4hiT1mTb2ZhlNHa/REmyktruNRMKo3/wqyUQi7LCU6YA1z8GGbPjBXQPsrHYM1LWEAZjtgHFzYNz88L2te4Ut2EwnTNw37OQ0fq8wZBsnQt2YMERTtWH46pqbiFQhhXgM9eTyvLqukxdXd/Dsyk2s2tjNpi1hOL/W1rndqeraVII9JzYxsbmW2a2NtDbVMH1sPZNa6pjcXMv4xhpa6tOMbUhTm0xAx2pY/QxsWQNrlkB2C6x/BV5fBD2bBy6qtgXqx8KWjTDlQGieApPfGl7DnLRf+HrjxDCEx0yHlulhAIuIyA5TiO+i2ruzvN7WxdI1Haza1M2a9m5eXN3OC292sK6jZ5ttWxtrmD6unulj6zl8j/HMam1kfGOafae0MHVMHWPq09v3Du1aH7aU21fBM38Kr7OueRZWPj5wQeP3ghnzoKEVJrwFphwATZPDU9d1Y9QSFhGpAIV4heXyAW+s7+Kp5Zt4ZsUm/vD8GjZ352jr7MEL7t5pqEmy18Qmjn7LRKaNqWPa2HrmTm5in8nNNNelh95J92ZY8SisewnefBpWPhG2tOl3e9Csd8D8i8LrwZPeChP3Ca8bN7QqpEVEdkEK8TJr785y75LVPP7GRp5ZuYkX3mynKxMO4FCTSnDAtBamj6vn8D1ms+fEJvac2MjMcQ001hb5owryYUCvXgKrnoCl98H6V8NryhB24pp+CLzz/8CsI2HsTGiYAE0TS/MNi4hIySjES6yjJ8dfXlrLo69t4LE3NvDU8k3kAyeZMObNHsf7D53B/tNaOGD6GPaZ3Ew6OcKezWtfgEd+DD0dsOE1WPbQ1sBO1Yet6YPOg/3PCK9Nj5mpVrWIyG5CIV4CG7sy3PHUKu5dsprFL7eRyQfUJBMcNHMMH33Xnrx77gTmzRlPTWoHbkVyh5V/hSdvgoev2/a1aQfD294Psw4Pn085SLc7iYjsxhTio+SZFZv4n2ff5MFX1/Pwq+HkbXtMaOS8+TM5+W1TefvMsdSlkzu3k5f/APd9KTxNDmEP71lHhqfGpxywc58tIiKxoxDfSWvau/nMLU/xwIvh1HP7TmnmkuPmcsxbJvL2mWNHZ8zg538H93817JQGcPTn4eDzw1u1RESkainEd8ITyzby8Z8/RltHhv9z/D6cf8QsWptG8d7nl+6Duz4T3qMNcMKXw/CuHzd6+xARkdhSiO+gGx96g6tvf5ZJzbX85hPv4IDpozgCdfcmuPcq+OsN4ahkx14Jh344HC9bREQkohDfAbc8uozP3/o0R+7Zyvc+dAjjG2uGf1Mx3OGxn8Kdn4YgB287F977JWiZNjqfLyIiuxWF+Ai9tq6Tz9/6NIfMGssNH5k/8lvCBpPrgV9dAC/dE8449f6fwn6nj85ni4jIbkkhPgL5wLn69mdJJRJ8928OGb0AX/Yw3PoxWP8yTD8UFvwuHDVNRERkCArxEbhh8Wvc/8JaLj5mb6aNHYWQ7WyDmy+A1/8XMDj8Y3DS1zQYi4iIFEUhXqQNnRn+9d4XmT9nPJ86YZ+d/8DNq+AH74At6+GQv4Xjr4aG8Tv/uSIiUjUU4kW69g8vsbk7xxdP33/n7/1e+QTceG7YC/3s/4S3nTMqNYqISHXRmJxFWN+Z4ecPvs6pB07d+VvJnrwJfnRs2JHt/F8rwEVEZIepJV6E+19YQzbv/P0799i5D3rwB3D352DifnDhbdA8ZVTqExGR6qQQL8Lil9tork1x0IyxO/4hL90bBvjso+C8m6CuZdTqExGR6qTT6UW48+lVHDx7HInEDl4L794Mv/koNE2Gc3+mABcRkVGhlvgwnl25ic5MnoNnjt3xD1n0HdiyAc69ARpbR602ERGpbmqJD+OOp1YBcO5hM3fsA7JbwhCfsA/sd8YoViYiItVOIT6Mv76+gQlNNUzf0cFd/vffIbcF3vUpDeIiIiKjSiE+hCBwlqzazHv338Fe5GueC1vhloQDzh7d4kREpOrpmvgQXlnXQXt3joNm7MC94V3r4YazINMJn1gMyfSo1yciItVNIT6EpWs6Adh36g70Jr/vauh4E87/DUzab3QLExERQafTh7R0TTsAe05sHNkbVy+Bx38OB34Q9j6uBJWJiIgoxIf00poOpo6po6VuhKfCf/8l8Dwcf1VpChMREUEhPqS/vrGBt0xpHtmb1r4IL94d3k7WMq00hYmIiKAQH5S7s2z9FppH2gpf9O/h4wn/NPpFiYiIFFCID2Jzdw6A1saa4t+0/tXwWvghF8K4OaUpTEREJKIQH8TyDV0A7DN5BKfTf3tx+HjUZaNfkIiISD8K8UG80RaG+FunFXl7WS4Dr/8FJu0PrXuVsDIREZGQQnwQL67uAKC1qcjT6Y//LHx816dKVJGIiMi2FOKD2LQlC8CUlrri3vDCXdA4UcOriohI2SjEB7Elm6exJkkqWcQh6t4ML/8B9j1Nk5yIiEjZKMQH8fKaDqYVO3PZS/8TDu6y3+mlLUpERKSAQnwQW7J5coEXt/GDP4D68bDHu0tblIiISAGF+CC6Mjn2ntQ0/IZtL8OKR+GIj2umMhERKSuF+CA2bckyoZie6U/eFD5qohMRESkzhfgA3J11HRkaa4qYqfW1v0D9OJh+aOkLExERKaAQH0BnJg9AMjFMT/P1r8Abi2D+RWWoSkREZFsK8QGsbe8BGP6a+DP/HT4e+IESVyQiIrI9hfgANkcDvYwfavKT9a/AIz+B8XtpmFUREamIIi76Vp+NUYiPqR+it/nCS6B9JVy4sExViYiIbEst8QH0zmDWMliIu8Nrf4Y574I931PGykRERLZSiA+hsXaQExWrnggfZx5etlpERET6U4gPYPOWHDDE6fSX7gsf9zmxTBWJiIhsTyE+gPbuLAmDxprkwBv0tsSnvr1cJYmIiGxHIT6Ajp4cLfVpbLAZyda9CM3TIFXkXOMiIiIloBAfQHt3jqbBrod3rgtD/JALyluUiIhIPwrxAfTk8tSnBzmV/vIfw8dxe5SvIBERkQEoxAewJZOnJjXIoVn/cvg494TyFSQiIjIAhfgAXm/rGnzc9BfvDlvhja3lLUpERKQfhfgA1rT3kBooxIMA1r8KTZPKX5SIiEg/CvEBNNYmBx7o5dUHoHsjHPTBstckIiLSn0J8AB3dOeZOat7+hWUPhY/7nVnegkRERAagEO9nSyZPZybPhOYB7gFfeh9MOVDXw0VEZJegEO9nXUc4l/iExtptXwjysPwRmH5oBaoSERHZnkK8n46ecNz05rp+18S3bAgfG8aXuSIREZGBKcT72RzNJd5c12/yk03LwsfJB5S5IhERkYEpxPtp7x6kJf7qn8LH1r3KXJGIiMjAFOL9dGbCEG+s7Tfs6vJHwscpB5a5IhERkYEpxPvZkskDbH+f+Iq/QrIWBpvZTEREpMwU4v10Z8MQr031a4lvXgHjNemJiIjsOhTi/XT1hXjBoWl/M3yccVgFKhIRERmYQryfTV1ZapKJbU+nb3gtfJz9jorUJCIiMhCFeD9L13Rs36lt9bPhozq1iYjILkQh3k99TZIt0Sn1Pj3t4WPjxPIXJCIiMoiShriZnWRmL5jZUjP73ACvzzKzP5rZ42b2lJmdUsp6itHRM8DkJ0tug6bJmoJURER2KSULcTNLAt8DTgb2B84zs/37bXYlcLO7Hwx8EPh+qeop1qYtWVrq+91e1vYKtEzX7WUiIrJLKWVLfD6w1N1fcfcMcBPQfw5PB1qi52OAlSWspygvre6goaYgxLs3Q88mmPPOyhUlIiIygNTwm+yw6cCyguXlwOH9trka+B8z+0egETi+hPUUpaEmST7wrStWPxM+TnlbZQoSEREZRKU7tp0H/NTdZwCnAD8zs+1qMrOLzOxRM3t07dq1JS2ooyfHHhMat654Mwrxmf3//hAREamsUob4CmBmwfKMaF2hjwA3A7j7YqAOmND/g9z9Onef5+7zJk4sXQ/xfOB0ZfK0FM5gtvH1cLjVsbNKtl8REZEdUcoQfwSYa2Z7mFkNYce1hf22eQM4DsDM9iMM8dI2tYfQO/lJQ03BfeLLHoKWaerUJiIiu5yShbi754CLgXuA5wh7oT9rZteY2RnRZp8CPmpmTwK/BBa4uw/8iaW3qSucSzwbBFtXZrdAMj3IO0RERCqnlB3bcPc7gTv7rftiwfMlwFGlrGEk1nX0ADB9bP3WlaufgUMXVKYgERGRIVS6Y9suJZsPTwI010V/22S3hI8JtcRFRGTXoxAv0DsNaV/HttVLwscZ8ypUkYiIyOAU4gVWb+4GCuYSb3spfBy/V4UqEhERGZxCvEBNNId4Khn1RH/z6fBx0n4VqkhERGRwCvECPbmwV3pT71zi+bC3OjWNg7xDRESkchTiBXqia+J16d7T6UuhYYLuERcRkV2SQrxAVyYM8W0Ge1GAi4jILkohXqA7G55O72uJL3tIY6aLiMguSyFeYH1nD2aQTESt70wHeDD0m0RERCpEIV6gvTtH36CvHWvCx9a9K1aPiIjIUBTiBVJJY2xDNNBL29LwcdYRlStIRERkCArxAi+v7WRcQ024sPzR8FEtcRER2UUpxAuMrU+zsSsTLvSO1jZm5uBvEBERqSCFeIFMPmCPCdHALl3rw8eahsoVJCIiMgSFeIGebLB13PRsF9SNqWxBIiIiQ1CIF+jO5alNR4dk/Sua+ERERHZpCvECnT05GnvHTc/1gOcrW5CIiMgQFOIFenIBtdFMZrS/CU2TK1uQiIjIEBTiBcIQT0IQAA7j96x0SSIiIoNSiBfo6smFk59kO8MVyXRlCxIRERmCQjwSBE5nJh/OJb5pebhywj6VLUpERGQICvFIe08OCE+ps/7VcOWYGRWsSEREZGgK8Uh3NuyJPn1cPXRGk5/oPnEREdmFKcQjvSFen05Ctjtc2Ty1ghWJiIgMTSEe6cqEId5Qk4T2VZBIQeOkClclIiIyOIV4ZJuW+PpXoGUaJFMVrkpERGRwRaWUmSWAg4BpwBbgGXdfU8rCyi0fOADJhMHqZ6F5WoUrEhERGdqQIW5mewH/FzgeeAlYC9QB+5hZF/AfwPXuHpS60FLLRSGeSlp4f3imo8IViYiIDG24lviXgR8A/+DuXviCmU0C/ga4ALi+NOWVTy4fhXgiAZlOmPPOClckIiIytCFD3N3PG+K1NcC/jXZBlZILwpMJyYRB9yaobalwRSIiIkPb4Y5tZvbe0Syk0rJRSzxteejZDDUNFa5IRERkaDvTO/0/R62KXUAmF7bEG7IbwxWpusoVIyIiUoThOrYtHOwloHX0y6mc3lvMGnIbwhWNEypYjYiIyPCG69j2LuB8oH9XbQPml6SiCunOhSFel9kYrmjdu3LFiIiIFGG4EH8Q6HL3B/q/YGYvlKakyujJhqfTazveCFfUj69gNSIiIsMbrnf6yUO89u7RL6dy+gZ76R2lraaxgtWIiIgMT8OuRrK9t5jle8IVtc0VrEZERGR4CvFIPrrFLJHZFK5QS1xERHZxCvFI77CriWwXWFIhLiIiuzyFeCQXBCQThuUzkKqtdDkiIiLDKjrEzezqoZbjLpMLSCcNct2QrKl0OSIiIsMaSUv8sWGWY607G4RziXdvhjqNmy4iIru+okPc3W8fajnusvmAdDIRtsTTGjddRER2fcMNu/odwAd73d0vGfWKKiSb9zDEO1brdLqIiMTCcCO2PVqWKnYBnT05UkmDTBf03isuIiKyCxtuxLbrC5fNrMHdu0pbUmWs6+gJZzLr3gRTD6x0OSIiIsMq6pq4mR1pZkuA56Plg8zs+yWtrMzG1KepSQDtq2DcnEqXIyIiMqxiO7b9G3Ai0Abg7k8Cu9XY6Zl8wJQmIMhqGlIREYmFkfROX9ZvVX6Ua6moXN5psOhbUu90ERGJgWJDfJmZvQNwM0ub2aeB50pYV9ll8wEtiehyfz5b2WJERESKUGyIfwz4JDAdWAm8PVrebXTn8tQnwpnMNNiLiIjEwXC3mAHg7uuAD5W4loraksnT1BjdEl/TVNliREREilBs7/Q9zex2M1trZmvM7Ldmtmepiyun7mxAXSpqiSfTlS1GRESkCMWeTr8RuBmYCkwDbgF+WaqiKmHFxi00BNE18YRCXEREdn3FhniDu//M3XPR18+BulIWVm4NNUnyuVy44LtVx3sREdlNDTd2+vjo6V1m9jngJsKx1D8A3Fni2soql3cmN1q40DipssWIiIgUYbiObY8RhnaUbvxDwWsO/L9SFFVu7k4mHzA23xauSNVWtiAREZEiDDd2+h7lKqSSenJhh7ZsEP2tktqtrhSIiMhuqqhbzADM7ABgfwquhbv7DaUoqtwy+TDEJ+TeDFfUj61cMSIiIkUqKsTN7CrgaMIQvxM4GfgLsFuEeDZqidfURPOI6z5xERGJgWJ7p58DHAe86e4fBg4CxpSsqjLLBeEgLymi+8TT9RWsRkREpDjFhvgWdw+AnJm1AGuAmaUrq7wyUUs85ZnwHnGzYd4hIiJSecVeE3/UzMYCPyLssd4BLC5VUeXWlQnvC69vf52w072IiMiur9ix0z8RPf2hmd0NtLj7U6Urq7zy0en0ZOM4WJOrcDUiIiLFGW6wl0OGes3d/zr6JZVfb+/05vaXYVxV3FUnIiK7geFa4t8a4jUHjh3FWiqm95p4UNMMG16rbDEiIiJFGm6wl2PKVUglbezKANCw8SWY/NYKVyMiIlKcYnun79bSyfAweE0T9GyucDUiIiLFUYiz9Zp4KtsOk/avcDUiIiLFUYgD2SjEk1vaoG5sZYsREREpUlEhbqHzzeyL0fIsM5tfxPtOMrMXzGxpNJXpQNuca2ZLzOxZM7txZOWPjlzeSZEjEWQgkaxECSIiIiNW7GAv3wcCwt7o1wDtwH8Dhw32BjNLAt8D3gssBx4xs4XuvqRgm7mE05ke5e4bzKwiE3l3Z/M0sSVc0OQnIiISE8WeTj/c3T8JdAO4+wagZpj3zAeWuvsr7p4BbgLO7LfNR4HvRZ+Hu68puvJRlA2cZusKF5qmVKIEERGRESs2xLNRy9oBzGwi9M4WMqjpwLKC5eXRukL7APuY2f+a2YNmdlKR9YyqIHCawr9PoK6lEiWIiIiMWLGn068FbgUmmdk/E85qduUo7X8u4TSnM4A/mdnb3H1j4UZmdhFwEcCsWbNGYbfbygfORIt2mRzuBIOIiMiuodix039hZo8RTkdqwFnu/twwb1vBtjOdzYjWFVoOPOTuWeBVM3uRMNQf6bf/64DrAObNmzfqM5QE7vTNW1bTONofLyIiUhLF9k6/Fhjv7t9z9+8WEeAQBvFcM9vDzGqADwIL+21zG2ErHDObQHh6/ZUiax81+cCpIRsu1Op0uoiIxEOx18QfA640s5fN7JtmNm+4N7h7DrgYuAd4DrjZ3Z81s2vM7Ixos3uANjNbAvwR+Iy7t43829g5eXfq6QkXUnXl3r2IiMgOKfZ0+vXA9WY2Hjgb+Bczm+Xuc4d5353Anf3WfbHguQOXR18VEwROo6ljm4iIxMtIR2zbG9gXmA08P/rlVEYucFLkw4VEurLFiIiIFKnYa+JfN7OXCAd6eQaY5+6nl7SyMlq9uXtriCeL7bAvIiJSWcUm1svAke6+rpTFVMq4hhqyfS1xhbiIiMTDkIllZvu6+/OEPc1nmdk2N2m7+19LWVy55N2Zk4z+PtHpdBERiYnhmp2XEw6y8q0BXnPCsdRjLwicLUS90lO1lS1GRESkSEOGuLtfFD092d27C18zs93mXqx8AElzSNWD2fBvEBER2QUU2zt9UZHrYilwJ2mBpiEVEZFYGe6a+BTCSUvqzexg6BudtAVoKHFtZZMPnBQBmEJcRETiY7hr4icCCwjHPf92wfp24PMlqqns8u6kzCEx0tvmRUREKme4a+K9I7Wd7e7/Xaaaym5TV5aEWuIiIhIzw51OP9/dfw7MMbPthkZ1928P8LbYyQUBuVxO18RFRCRWhjud3jsvZ1OpC6mk+nSSvVNr2XrJX0REZNc33On0/4gev1SeciojFzibk2NgywuVLkVERKRoIxk7vcXM0mb2ezNba2bnl7q4cgncqSULE95S6VJERESKVmx37BPcfTNwGvAa4WxmnylVUeWWD5xZwXJNfiIiIrFSbIj3ptupwC3uvqlE9VREPoDNNgZ62itdioiISNGKbXreYWbPA1uAj5vZRKB7mPfERuBOmhyM37PSpYiIiBStqJa4u38OeAfhPOJZoBM4s5SFlVMucGrpgWRNpUsREREpWlEtcTNLA+cD77ZwgpAHgB+WsK6yWra+iyn5VRDsV+lSREREilbsNfEfAIcC34++DonW7RYmNteywcZAkK90KSIiIkUr9pr4Ye5+UMHyH8zsyVIUVAnuTjrh0DS50qWIiIgUrdiWeN7M9updMLM9gd2m2ZoPnIRrKlIREYmXYlvinwH+aGavEI5NOhv4cMmqKrPAIYlCXERE4mXYEI9uJ9sEzAcmRatfcPeeUhZWToE7CfKaxUxERGJlyNPpZvb3wLPAd4AngDnu/tTuFOAQnk6vDbohoRHbREQkPoZLrcuAt7r72ug6+C+AhSWvqswChzRZyHRUuhQREZGiDdexLePuawHc/RWgtvQllV8QePikZreecVVERHYzw7XEZ5jZtYMtu/slpSmrvPJBED6pa6lsISIiIiMwXIj3n6nssVIVUkkrN3ZCEnVsExGRWBkyxN39+nIVUkmTG1PhdC66xUxERGJkuN7pPzKzAwZ5rdHM/s7MPlSa0srHPDqdrhAXEZEYGe50+veAL5rZ24BngLVAHTAXaAF+QthjPd56x0zX6XQREYmR4U6nPwGca2ZNwDxgKuGc4s+5+wulL688LMiET3SfuIiIxEhRqeXuHcD9pS2lchq9M3xSN6ayhYiIiIxAsROg7NZSQXf4pKahsoWIiIiMgEIcqPcoxNMKcRERiY8RhbiZ7ZYpF2S3hE9qGitbiIiIyAgUFeJm9g4zWwI8Hy0fZGbfL2llZVRL1LEtVVfZQkREREag2Jb4vwInAm0A7v4k8O5SFVVujZYNnyRrKluIiIjICBR9Ot3dl/VblR/lWiomSS58opa4iIjESLE3Ri8zs3cAbmZp4FLgudKVVT7uTkIjtomISAwV2xL/GPBJYDqwAng78IkS1VRW7pAgCnGzyhYjIiIyAsW2xN/i7tuMkW5mRwH/O/ollVfgTtJ6Q1wtcRERiY9iW+LfKXJd7AQOCTxc0Ol0ERGJkSFb4mZ2JPAOYKKZXV7wUgvhDNyxF7iTRC1xERGJn+FOp9cATdF2zQXrNwPnlKqocuu7Jq6WuIiIxMhws5g9ADxgZj9199fLVFNZBe5MsfXhglriIiISI8V2bOsys28AbyWcTxwAdz+2JFWVUeCw2aPRZJPpyhYjIiIyAsV2bPsF4ZCrewBfAl4DHilRTWUVXhOPOrZpxDYREYmRYkO81d3/E8i6+wPu/ndA7FvhAB5AsnfwOV0TFxGRGCn2dHo0uDirzOxUYCUwvjQllZd6p4uISFwVG+JfNrMxwKcI7w9vAS4rVVHllA0CkpbHMSyh6dVFRCQ+igpxd78jeroJOAb6RmyLP4dxdFS6ChERkREbbrCXJHAu4Zjpd7v7M2Z2GvB5oB44uPQlllbgkCaH9XZuExERiYnhWuL/CcwEHgauNbOVwDzgc+5+W4lrKwvH6SFNNtWIbjATEZE4GS7E5wEHuntgZnXAm8Be7t5W+tLKI3BIEZBL1inERUQkVobryZVxDyfbdvdu4JXdKcAhnE88SR63Yvv4iYiI7BqGS659zeyp6LkBe0XLBri7H1jS6srAHdKWx3V7mYiIxMxwIb5fWaqoIHeYY28qxEVEJHaGmwBlt5z0pFDgzjofw4GZlZUuRUREZESqfnSTwJ0UeTobZ1e6FBERkRGp+hB3IEWeIKGObSIiEi9Fh7iZ1ZvZW0pZTCV41BJX73QREYmbokLczE4HngDujpbfbmYLS1hX2bjDHolVuFriIiISM8W2xK8G5gMbAdz9CcK5xWMvcFjrY6nrXlvpUkREREak2BDPuvumfut2i8HGnXAq0q4mdWwTEZF4KfYc8rNm9jdA0szmApcAi0pXVvkEASRwPKH7xEVEJF6KbYn/I/BWoAe4kXBK0stKVFNZOU6CAKzqO+qLiEjMFNsS39fdrwCuKGUxlZDJBTQR4ApxERGJmWKT61tm9pyZ/ZOZHVDSisrMgQQBOVeIi4hIvBSVXO5+DHAMsBb4DzN72syuHO59ZnaSmb1gZkvN7HNDbHe2mbmZzSu68lFihCGeTumauIiIxEvRzU93f9PdrwU+RnjP+BeH2t7MksD3gJOB/YHzzGz/AbZrBi4FHiq+7NETRLOYoQlQREQkZood7GU/M7vazJ4GvkPYM33GMG+bDyx191fcPQPcBJw5wHb/BPwL0F182aPH3Zlh60gE2UrsXkREZIcV2xL/CeFALye6+9Hu/gN3XzPMe6YDywqWl0fr+pjZIcBMd/9dkXWMusChzZsxz1eqBBERkR1SVO90dz9ytHdsZgng28CCIra9CLgIYNasWaNaR+CO4WTqJ47q54qIiJTakC1xM7s5enzazJ4q+HrazJ4a5rNXADMLlmdE63o1AwcA95vZa8ARwMKBOre5+3XuPs/d502cOLphG7iTwEGDvYiISMwM1xK/NHo8bQc++xFgrpntQRjeHwT+pvfFaBjXCb3LZnY/8Gl3f3QH9rXD3IkGe7Fy7lZERGSnDdkSd/dV0dNPuPvrhV/AJ4Z5bw64GLgHeA642d2fNbNrzOyM0Sh+NIQh7pha4iIiEjPFjtj2XuD/9lt38gDrtuHudwJ39ls34K1p7n50kbWMqsDDCVA07KqIiMTNkCFuZh8nbHHv2e8aeDPwv6UsrFz6ronrPnEREYmZ4VriNwJ3AV8FCkdca3f39SWrqozcodayOp0uIiKxM1yIu7u/Zmaf7P+CmY3fHYI8CML7wxP5iow1IyIissOKaYmfBjxGOFdIYRduB/YsUV1l4/lc+CSRrmwhIiIiIzRkiLv7adHjHuUpp/w2dW0BIFc7rsKViIiIjEyxY6cfZWaN0fPzzezbZja6Q6dVSHM6OrmQKLajvoiIyK6h2PuqfgB0mdlBwKeAl4GflayqMvJo4pNkSiEuIiLxUmyI59zdCWch+667f4/wNrP4CwIA9U4XEZHYKbb52W5m/w+4AHhXNHnJ7tETLBf1StewqyIiEjPFtsQ/APQAf+fubxJOZvKNklVVRtZ7i1lOt5iJiEi8FBXiUXD/AhhjZqcB3e5+Q0krKxMnPJ0e1LdWuBIREZGRKbZ3+rnAw8D7gXOBh8zsnFIWVi7ed01cY6eLiEi8FHtN/ArgMHdfA2BmE4H7gF+XqrBy8cABMF0TFxGRmCm2+ZnoDfBI2wjeu0sLO92jWcxERCR2im2J321m9wC/jJY/QL8pRuMr6timEBcRkZgpKsTd/TNm9v8B74xWXefut5aurPLZ2hDX6XQREYmX4eYTnwt8E9gLeBr4tLuvKEdh5eLRLWamlriIiMTMcMn1E+AO4GzCmcy+U/KKymzrNXG1xEVEJF6GO53e7O4/ip6/YGZ/LXVBZReFuG4xExGRuBkuxOvM7GC2ziNeX7js7rEP9e5sOJ+4TqeLiEjcDBfiq4BvFyy/WbDswLGlKKqcsrnomjg6nS4iIvEyZIi7+zHlKqRS6tNhCzyZVEtcRETipeqTq693uq6Ji4hIzFR9cvV2TteIbSIiEjdKLg8nQNGIbSIiEjfFzmJmZna+mX0xWp5lZvNLW1p5WL4nfJJKV7YQERGRESq2+fl94EjgvGi5HfheSSoqs0SQAcBS9RWuREREZGSKnQDlcHc/xMweB3D3DWZWU8K6ykeDvYiISEwVm1xZM0sS3hveO594ULKqyqn3mngiWeFCRERERqbYEL8WuBWYZGb/DPwF+ErJqiojD8IQV0tcRETiptipSH9hZo8BxxEOuXqWuz9X0srKJgpx9U4XEZGYKSrEzWwW0AXcXrjO3d8oVWHlYtHpdN0nLiIicVNsx7bfEV4PN6AO2AN4AXhrieoqG1eIi4hITBV7Ov1thctmdgjwiZJUVG4KcRERiakdSq5oCtLDR7mWyggU4iIiEk/FXhO/vGAxARwCrCxJReWmlriIiMRUsdfEmwue5wivkf/36JdTftnNa8InCnEREYmZYUM8GuSl2d0/XYZ6yq6moSl8ohAXEZGYGTK5zCzl7nngqDLVU3Z9t5il6ypbiIiIyAgN1xJ/mPD69xNmthC4BejsfdHdf1PC2srCeycUV0tcRERipthr4nVAG3AsW+8XdyD2IY7nw0eFuIiIxMxwIT4p6pn+DFvDu5eXrKoyMt1iJiIiMTVciCeBJrYN7167RYh7X0tcs5iJiEi8DBfiq9z9mrJUUil918QH+jtFRERk1zXcOeTdP9kCXRMXEZF4Gi65jitLFZWkEdtERCSmhkwud19frkIqpTnXFj5J6Jq4iIjES9U3P9f2RN0CkjWVLURERGSEqj7EG2vT4ZNEsbfMi4iI7BqqPsS975r47t+HT0REdi9VH+LmAXkdBhERiSGll+fxKriTTkREdj8KcXeFuIiIxJJC3AMCdHuZiIjEj0LcA1yd2kREJIaqPsTNAwIdBhERiaGqT6+O7oyuiYuISCxVfYjPTG3cOpOZiIhIjFR9iHdaA410VboMERGREav6EHeMjcnWSpchIiIyYlUf4jjoZLqIiMSRQhzHdRhERCSGqj69jADUO11ERGKo6kMcdw32IiIisVT1IW5o7HQREYmnqg9xUEtcRETiqepD3FzXxEVEJJ4U4qDe6SIiEktVn15GoGviIiISS1Uf4rjrbLqIiMRSSUPczE4ysxfMbKmZfW6A1y83syVm9pSZ/d7MZpeynoFpsBcREYmnkqWXmSWB7wEnA/sD55nZ/v02exyY5+4HAr8Gvl6qegaTwFFTXERE4qiUTdD5wFJ3f8XdM8BNwJmFG7j7H929dwqxB4EZJaxnYBrsRUREYqqUIT4dWFawvDxaN5iPAHeVsJ4BabAXERGJq1SlCwAws/OBecB7Bnn9IuAigFmzZo3qvqf5m+wih0FERGREStkSXwHMLFieEa3bhpkdD1wBnOHuPQN9kLtf5+7z3H3exIkTR7XItYynJbduVD9TRESkHEoZ4o8Ac81sDzOrAT4ILCzcwMwOBv6DMMDXlLCWQRkB62vLfyleRERkZ5UsxN09B1wM3AM8B9zs7s+a2TVmdka02TeAJuAWM3vCzBYO8nElY+646RYzERGJn5JeDHb3O4E7+637YsHz40u5/2KE84krxEVEJH6qPr0SqCUuIiLxVPXpFY6dXvWHQUREYqjq08vcQYO9iIhIDFV9iCfUEhcRkZiq+vQyHHRNXEREYqjq08sINHa6iIjEUtWHeJJALXEREYmlqk+venrIW7rSZYiIiIxY1Yd4koB8orbSZYiIiIxY1Yc46tgmIiIxVdXp5e4kPNCIbSIiEktVnV7ukDAN9iIiIvFU1SHekwswnESiqg+DiIjEVFWnV+BOQtfERUQkpqo6vZxwxDYNuyoiInFU1enlaomLiEiMVXV6BR4Ou6qObSIiEkdVHeI4UUtcIS4iIvFT1SHuuK6Ji4hIbFV1ernDeOvQNXEREYmlqk6vwB2AhkxbhSsREREZuaoOcQdynqCjflqlSxERERmx6g5xJ7wqbslKlyIiIjJi1R3iOAkCSCjERUQkfqo7xAMnaY7rFjMREYmh6g7xqGMbqCUuIiLxU90hHuTCJ5rFTEREYqiq0ysIgvCJOraJiEgMVXWIu+fDJxrsRUREYqi60ysfhrg6tomISBxVdYhbvgcAT9RUuBIREZGRq+oQ7x12VafTRUQkjqo6vYJAIS4iIvFV1emVz4e3mJluMRMRkRiq6vTqbYkn1BIXEZEYqvL0Cu8TN/VOFxGRGKrqEA/yvdfEFeIiIhI/VR3iHrXENeyqiIjEUXWnV++wq6glLiIi8VPVId47i5mpY5uIiMRQVaeXe+8EKGqJi4hI/FR5iKslLiIi8VXV6aWWuIiIxFl1h3iglriIiMRXVafX1vnE1RIXEZH4qeoQp2+sF4W4iIjET6rSBVRUdgugu8RFJD42b97MmjVryGazlS5FRlFjYyMzZswgMcLBx6o6xD1qgSeDngpXIiIyvM2bN7N69WqmT59OfX29ziLuJoIgYMWKFaxbt45JkyaN6L1VfTq9t3d6rmZsZQsRESnCmjVrmD59Og0NDQrw3UgikWDy5Mls2rRp5O8tQT3x0TcVqf4ziMiuL5vNUl9fX+kypATS6TS5XG7E76vqEA9cs5iJSLyoBb572tGfa1WHeFcm7BiSV9c2EREZJU1NTbzyyitl2VdVh3htKgzvkfYGFBGR7c2ZM4f6+nqampqYMmUKCxYsoKOjY5ttFi1axLHHHktzczNjxozh9NNPZ8mSJdtss3nzZi677DJmzZpFU1MTe+21F5dddhnr1q0raf33338/M2bM2OnP6ejoYM899xyFioZX1ell0en0lEJcRGRU3H777XR0dPDEE0/w+OOP89WvfrXvtcWLF3PCCSdw5plnsnLlSl599VUOOuggjjrqqL6WayaT4bjjjuPZZ5/l7rvvZvPmzSxevJjW1lYefvjhSn1bfXbkunUpVXV6ua6Ji4iUxJQpUzjxxBN54okn+tZ99rOf5cILL+TSSy+lubmZ8ePH8+Uvf5kjjjiCq6++GoAbbriBN954g1tvvZX999+fRCLBpEmT+MIXvsApp5wy4L4WLVrEYYcdxpgxYzjssMNYtGhR32tHH300X/jCFzjqqKNobm7mhBNOGLBF39nZycknn8zKlStpamqiqamJlStXcvXVV3POOedw/vnn09LSwk9/+lMefvhhjjzySMaOHcvUqVO5+OKLyWQyfZ9lZixduhSABQsW8MlPfpJTTz2V5uZmDj/8cF5++eVROMIhhTjqKCIiMtqWL1/OXXfdxd577w1AV1cXixYt4v3vf/9225577rnce++9ANx3332cdNJJNDU1FbWf9evXc+qpp3LJJZfQ1tbG5ZdfzqmnnkpbW1vfNjfeeCP/9V//xZo1a8hkMnzzm9/c7nMaGxu56667mDZtGh0dHXR0dDBt2jQAfvvb33LOOeewceNGPvShD5FMJvnXf/1X1q1bx+LFi/n973/P97///UFrvOmmm7jqqqvYsGEDe++9N1dccUVR31sxqnuwl74QT1a4EhGRkfvS7c+yZOXmku5j/2ktXHX6W4ve/qyzzsLM6Ojo4Nhjj+VLX/oSEIZtEARMnTp1u/dMnTq1r3Xc1tbGoYceWvT+fve73zF37lwuuOACAM477zyuvfZabr/9dhYsWADAhz/8YfbZZx8g/INh4cKFRX8+wJFHHslZZ50FQH19/Tb1zZkzh3/4h3/ggQce4LLLLhvw/e973/uYP38+AB/60Ie4/PLLR7T/oVR1S5y+CVAqW4aIyO7itttuo729nfvvv5/nn3++L5zHjRtHIpFg1apV271n1apVTJgwAYDW1tYBtxnMypUrmT179jbrZs+ezYoVK/qWp0yZ0ve8oaFhu852w5k5c+Y2yy+++CKnnXYaU6ZMoaWlhc9//vNDdrrb2f0Ppcpb4tETnU4XkRgaSQu53N7znvewYMECPv3pT3PbbbfR2NjIkUceyS233MIxxxyzzbY333wzxx13HADHH388V155JZ2dnTQ2Ng67n2nTpvH6669vs+6NN97gpJNOGnHNg11a7b/+4x//OAcffDC//OUvaW5u5t/+7d/49a9/PeL9jYYqb4n3jthW3YdBRKQULrvsMu69916efPJJAL72ta9x/fXXc+2119Le3s6GDRu48sorWbx4MVdddRUAF1xwATNnzuTss8/m+eefJwgC2tra+MpXvsKdd9653T5OOeUUXnzxRW688UZyuRy/+tWvWLJkCaeddtqI6508eTJtbW3DDn/a3t5OS0sLTU1NPP/88/zgBz8Y8b5GS3WnVzR2ulriIiKjb+LEiVx44YVcc801ALzzne/knnvu4Te/+Q1Tp05l9uzZPP744/zlL39h7ty5ANTW1nLfffex77778t73vpeWlhbmz5/PunXrOPzww7fbR2trK3fccQff+ta3aG1t5etf/zp33HFH3+n5kdh3330577zz2HPPPRk7diwrV64ccLtvfvOb3HjjjTQ3N/PRj36UD3zgAyPe12ixvtusYmLevHn+6KOPjspnPfiH2zjiT3/LG6ffzKxDTxyVzxQRKZXnnnuO/fbbr9JlSIkM9vM1s8fcfd5A76nulnig+8RFRCS+qjrEnfB0uu4TFxGROKrqEEeDvYiISIxVdYgncl3hE/VOFxGRGKrq9OqdAMV81xrQXkREpBhVHeK9/fK9pqWidYiIiOyIqg7xvmFXExo7XURE4qe6QzwIe6cnFOIiIhJDJQ1xMzvJzF4ws6Vm9rkBXq81s19Frz9kZnNKWc92+iZAUYiLiEj8lCzELZzf83vAycD+wHlmtn+/zT4CbHD3vYF/Bf6lVPUMKAhD3JIKcRGRnTVnzhzq6+tpampiypQpLFiwYLsZuxYtWsSxxx5Lc3MzY8aM4fTTT2fJkiXbbLN582Yuu+wyZs2aRVNTE3vttReXXXbZkDOFjYb777+fGTNmjMpnHX300fz4xz8elc8aSilb4vOBpe7+irtngJuAM/ttcyZwffT818BxVs6btqOx010tcRGRUXH77bfT0dHBE088weOPP85Xv/rVvtcWL17MCSecwJlnnsnKlSt59dVXOeiggzjqqKN45ZVXAMhkMhx33HE8++yz3H333WzevJnFixfT2trKww8/XKlva5dVyhCfDiwrWF4erRtwG3fPAZuA1hLWtI1EdDo9oZa4iMiomjJlCieeeCJPPPFE37rPfvazXHjhhVx66aU0Nzczfvx4vvzlL3PEEUdw9dVXA3DDDTfwxhtvcOutt7L//vuTSCSYNGkSX/jCFzjllFMG3NeiRYs47LDDGDNmDIcddhiLFi3qe+3oo4/mC1/4AkcddRTNzc2ccMIJA7boOzs7Ofnkk1m5ciVNTU00NTWxcuVKgiDga1/7GnvttRetra2ce+65rF+/HoDu7m7OP/98WltbGTt2LIcddhirV6/miiuu4M9//jMXX3wxTU1NXHzxxaN3YPuJRcc2M7vIzB41s0fXrl07ap+bbp7Ii6l9SNXUj9pniogILF++nLvuuou9994bgK6uLhYtWsT73//+7bY999xzuffeewG47777OOmkk2hqaipqP+vXr+fUU0/lkksuoa2tjcsvv5xTTz2Vtra2vm1uvPFG/uu//os1a9aQyWT45je/ud3nNDY2ctdddzFt2jQ6Ojro6Ohg2rRpfOc73+G2227jgQceYOXKlYwbN45PfvKTAFx//fVs2rSJZcuW0dbWxg9/+EPq6+v553/+Z971rnfx3e9+l46ODr773e+O+PgVK1WyT4YVwMyC5RnRuoG2WW5mKWAM0NZvG9z9OuA6CGcxG60C3/7ev4H3/s1ofZyISHnd9Tl48+nS7mPK2+DkrxW9+VlnnYWZ0dHRwbHHHsuXvvQlIAzbIAiYOnXqdu+ZOnVqX+u4ra2NQw89tOj9/e53v2Pu3LlccMEFAJx33nlce+213H777SxYsACAD3/4w+yzzz5A+AfDwoULi/78H/7wh3z3u9/tu1Z+9dVXM2vWLH72s5+RTqdpa2tj6dKlHHjggSOqe7SUsiX+CDDXzPYwsxrgg0D/I7cQ+Nvo+TnAHzxuc6OKiEif2267jfb2du6//36ef/75vnAeN24ciUSCVatWbfeeVatW9c3/3draOuA2g1m5ciWzZ8/eZt3s2bNZsWJrm3HKlCl9zxsaGrbrbDeU119/nfe9732MHTuWsWPHst9++5FMJlm9ejUXXHABJ554Ih/84AeZNm0an/3sZ8lms0V/9mgoWUvc3XNmdjFwD5AEfuLuz5rZNcCj7r4Q+E/gZ2a2FFhPGPQiIlKMEbSQy+0973kPCxYs4NOf/jS33XYbjY2NHHnkkdxyyy0cc8wx22x78803c9xxxwFw/PHHc+WVV9LZ2UljY+Ow+5k2bRqvv/76NuveeOMNTjrppBHXPFC/6pkzZ/KTn/yEo446asD3XHXVVVx11VW89tprnHLKKbzlLW/hIx/5SNkm1irpNXF3v9Pd93H3vdz9n6N1X4wCHHfvdvf3u/ve7j7f3V8pZT0iIlI+l112Gffeey9PPvkkAF/72te4/vrrufbaa2lvb2fDhg1ceeWVLF68mKuuugqACy64gJkzZ3L22Wfz/PPPEwQBbW1tfOUrX+HOO+/cbh+nnHIKL774IjfeeCO5XI5f/epXLFmyhNNOO23E9U6ePJm2tjY2bdrUt+5jH/sYV1xxRd8fCmvXruW3v/0tAH/84x95+umnyefztLS0kE6nSSQSfZ/V2+O+lGLRsU1EROJn4sSJXHjhhVxzzTUAvPOd7+See+7hN7/5DVOnTmX27Nk8/vjj/OUvf2Hu3LkA1NbWct9997Hvvvvy3ve+l5aWFubPn8+6des4/PDDt9tHa2srd9xxB9/61rdobW3l61//OnfccUff6fmR2HfffTnvvPPYc889GTt2LCtXruTSSy/ljDPO4IQTTqC5uZkjjjiChx56CIA333yTc845h5aWFvbbbz/e85739F2bv/TSS/n1r3/NuHHjuOSSS3b0EA7L4nYJet68ef7oo49WugwRkbJ77rnn2G+//SpdhpTIYD9fM3vM3ecN9B61xEVERGJKIS4iIhJTCnEREZGYUoiLiIjElEJcRCRG4tYZWYqzoz9XhbiISEyk02m2bNlS6TKkBLLZLKnUyMdfU4iLiMTEpEmTWLFiBV1dXWqR70aCIGD16tWMGTNmxO8t5QQoIiIyilpaWoBwvPByj9EtpdXY2LhDA9QoxEVEYqSlpaUvzEV0Ol1ERCSmFOIiIiIxpRAXERGJKYW4iIhITMVuFjMzWwu8PuyGxZsArBvFz6tWOo47T8dw5+kY7jwdw5032sdwtrtPHOiF2IX4aDOzRweb4k2Kp+O483QMd56O4c7TMdx55TyGOp0uIiISUwpxERGRmFKIw3WVLmA3oeO483QMd56O4c7TMdx5ZTuGVX9NXEREJK7UEhcREYmpqglxMzvJzF4ws6Vm9rkBXq81s19Frz9kZnMqUOYurYhjeLmZLTGzp8zs92Y2uxJ17sqGO4YF251tZm5m6iU8gGKOo5mdG/17fNbMbix3jbu6Iv4/zzKzP5rZ49H/6VMqUeeuysx+YmZrzOyZQV43M7s2Or5PmdkhJSnE3Xf7LyAJvAzsCdQATwL799vmE8APo+cfBH5V6bp3pa8ij+ExQEP0/OM6hiM/htF2zcCfgAeBeZWue1f7KvLf4lzgcWBctDyp0nXvSl9FHsPrgI9Hz/cHXqt03bvSF/Bu4BDgmUFePwW4CzDgCOChUtRRLS3x+cBSd3/F3TPATcCZ/bY5E7g+ev5r4DgzszLWuKsb9hi6+x/dvStafBCYUeYad3XF/DsE+CfgX4DuchYXI8Ucx48C33P3DQDuvqbMNe7qijmGDvROlzYGWFnG+nZ57v4nYP0Qm5wJ3OChB4GxZjZ1tOuolhCfDiwrWF4erRtwG3fPAZuA1rJUFw/FHMNCHyH8K1S2GvYYRqfcZrr778pZWMwU829xH2AfM/tfM3vQzE4qW3XxUMwxvBo438yWA3cC/1ie0nYbI/2duUM0n7iMOjM7H5gHvKfStcSJmSWAbwMLKlzK7iBFeEr9aMIzQn8ys7e5+8ZKFhUz5wE/dfdvmdmRwM/M7AB3DypdmGxVLS3xFcDMguUZ0boBtzGzFOHpo7ayVBcPxRxDzOx44ArgDHfvKVNtcTHcMWwGDgDuN7PXCK+jLVTntu0U829xObDQ3bPu/irwImGoS6iYY/gR4GYAd18M1BGOCS7FKep35s6qlhB/BJhrZnuYWQ1hx7WF/bZZCPxt9Pwc4A8e9U4QoIhjaGYHA/9BGOC6Brm9IY+hu29y9wnuPsfd5xD2KzjD3R+tTLm7rGL+P99G2ArHzCYQnl5/pYw17uqKOYZvAMcBmNl+hCG+tqxVxttC4MKol/oRwCZ3XzXaO6mK0+nunjOzi4F7CHtl/sTdnzWza4BH3X0h8J+Ep4uWEnZW+GDlKt71FHkMvwE0AbdEfQLfcPczKlb0LqbIYyjDKPI43gOcYGZLgDzwGXfXmbVIkcfwU8CPzOz/EHZyW6CGzVZm9kvCPxQnRP0GrgLSAO7+Q8J+BKcAS4Eu4MMlqUM/ExERkXiqltPpIiIiux2FuIiISEwpxEVERGJKIS4iIhJTCnEREZGYUoiLiIjElEJcYs/M8mb2RMHXnCG27RiF/f3UzF6N9vXXaEjKkX7Gj81s/+j55/u9tmhna4w+p/e4PGNmt5vZ2GG2f/uOTDdpZlPN7I7o+dFmtina73NmdtUOfN4ZvVNjmtlZvccpWr4mGhVwp0Q/w3OG2eb+kYyWF33vdxSx3YBTWJrZN83s2GL3JwIKcdk9bHH3txd8vVaGfX7G3d8OfI5wlLoRcfe/d/cl0eLn+732jp0vD9h6XA4gHMDok8Ns/3bCwSlG6nLgRwXLf46OzTzCCTRGNI+yuy90969Fi2cRToPZ+9oX3f2+HahxV/JTYKAJWb5D+O9JpGgKcdntmFmTmf0+aiU/bWbbTfcZtR7/VNBSfVe0/gQzWxy99xYzaxpmd38C9o7ee3n0Wc+Y2WXRukYz+52ZPRmt/0C0/n4zm2dmXwPqozp+Eb3WET3eZGanFtT8UzM7x8ySZvYNM3vEzJ4ys38o4rAsJppByczmR9/j42a2yMzeEg29eQ3wgaiWD0S1/8TMHo62HWjaVICzgbv7r3T3TuAxYO+olf9gVO+tZjYuquUSM1sSrb8pWrfAzL5rZu8AzgC+EdW0V8ExOMnMbik4Nn2t4JH+DM3si9GxfMbMrjPbZgriCwr+jcyPti/2uAxosCks3f11oNXMpozk86TKlXridH3pq9RfhMNqPhF93Uo4nHBL9NoEwmEPe0cn7IgePwVcET1PEk4+MoEwlBuj9f8X+OIA+/spcE70/P3AQ8ChwNNAI+HQs88CBxMG3I8K3jsmerwfmFdYU8E2vTW+D7g+el5DOK1hPXARcGW0vhZ4FNhjgDo7Cr6/W4CTouUWIBU9Px747+j5AuC7Be//CnB+9Hws4SQijf32sQfwWMHy0cAd0fNW4DXgrcBTwHui9dcA/xY9XwnU9u6jfx2Fx7pwOfoZv1Hws/oBcP4O/gzHF6z/GXB6wc/oR9HzdwPPDHVc+n3v84AfD/Fvdk7v5/Vb/yPg7Er/n9JXfL6qYux02e1t8fD0LQBmlga+YmbvBgLCFuhk4M2C9zwC/CTa9jZ3f8LM3kN46vZ/o8ZYDWELdiDfMLMrCSeE+AjhRBG3etj6xMx+A7yLsIX6LTP7F8Jf8H8ewfd1F/DvZlZLePr1T+6+xcxOAA4suKY7hnCGrlf7vb/ezJ6Ivv/ngHsLtr/ezOYSjomdHmT/JwBnmNmno+U6YFb0Wb2msv2kGO8ys8cJj/3XCGcUG+vuD0SvX0/4RwWE4f4LM7uNcNKSong49vfdwOlm9mvgVOCzhNPfFvsz7HWMmX0WaADGE/4Bdnv02i+j/f3JzFos7Fcw2HEprO9R4O+L/X4KrAGm7cD7pEopxGV39CFgInCou2ctnNazrnCD6Jfyuwl/+f/UzL4NbADudffzitjHZ9z9170LZnbcQBu5+4vRNeFTgC+b2e/d/Zpivgl37zaz+4ETgQ8AN/XuDvhHd79nmI/Y4u5vN7MGwokuPglcC/wT8Ed3f5+FnQDvH+T9RtgqfGGofdDv2BJeEz+t70PMxgzx/lMJW7mnA1eY2duG2La/m4CLCU9NP+ru7dGp8GJ/hphZHfB9wrMiy8zsarb9fvpPLuEMclzMbPIIah9MHeExFSmKronL7mgMsCYK8GOA2f03MLPZwGp3/xHwY+AQwqk/jzKz3mvcjWa2T5H7/DNwlpk1mFkj4anwP5vZNKDL3X9OOMvbQJ28stEZgYH8inD2o95WPYSB/PHe95jZPtE+B+TuXcAlwKfMLEV4fHrnNV5QsGk74WWFXvcA/9h7jdjCqWb7e5Hw1PCg3H0TsMGifgfABcADZpYAZrr7HwlPe48hvBRRqH9NhR4gPJ4fZesfOCP9GfYG9rro2nn/Huu9fRjeSTiV5CaKOy47ah/gmWG3EokoxGV39Atgnpk9DVwIPD/ANkcDT0anfT8A/Lu7ryUMtV+a2VOEp2H3LWaH7v5XwuusDxNeI/+xuz8OvA14ODqtfRXw5QHefh3wlEUd2/r5H8JTxPe5eyZa92NgCfBXC29T+g+GOasW1fIUcB7wdeCr0fde+L4/Avv3dmwjbLGno9qejZb7f24n8HJvaA7hbwkvQTxF2Av+GsJr9T+Pfk6PA9e6+8Z+77sJ+EzUgWyvfvvOA3cAJ0ePjPRnGO3vR4TBeQ/hZZZC3dFx+iHhZRMo4rhY2GnxxwPt08IpLBcDbzGz5Wb2kWh9mrCTpOaPl6JpKlIR2Slm9j7CSxdXVrqWOIuO4yHu/oVK1yLxoWviIrJT3P1WM2utdB27gRTwrUoXIfGilriIiEhM6Zq4iIhITCnERUREYkohLiIiElMKcRERkZhSiIuIiMTU/w9TweIRFipf8QAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score\n",
    "from sklearn.metrics import plot_roc_curve, roc_auc_score\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "def estimate_model(clf: GridSearchCV):\n",
    "    \"\"\" 测试模型，输出模型的各项得分，ROC 图 \"\"\"\n",
    "    # 利用训练好的模型对训练集、测试集进行预测\n",
    "    y_train_pred = clf.predict(X_train)\n",
    "    y_test_pred  = clf.predict(X_test)\n",
    "\n",
    "    accu_train  = accuracy_score(y_train, y_train_pred)\n",
    "    accu_test   = accuracy_score(y_test, y_test_pred)\n",
    "\n",
    "    preci_train = precision_score(y_train, y_train_pred)\n",
    "    preci_test  = precision_score(y_test, y_test_pred)\n",
    "\n",
    "    recall_train= recall_score(y_train, y_train_pred)\n",
    "    recall_test = recall_score(y_test, y_test_pred)\n",
    "\n",
    "    f1_train = f1_score(y_train, y_train_pred)\n",
    "    f1_test  = f1_score(y_test, y_test_pred)\n",
    "\n",
    "    roc_area_train = roc_auc_score(y_train, clf.predict_proba(X_train)[:,1])\n",
    "    roc_area_test  = roc_auc_score(y_test, clf.predict_proba(X_test)[:,1])\n",
    "\n",
    "    print(F\"训练集上的准确率：{accu_train:.5f},\\t测试集上的准确率：{accu_test:.5f}\")\n",
    "    print(F\"训练集上的精度：  {preci_train:.5f},\\t测试集上的精度：{preci_test:.5f}\")\n",
    "    print(F\"训练集上的召回率：{recall_train:.5f},\\t测试集上的召回率：{recall_test:.5f}\")\n",
    "    print(F\"训练集上的F1得分：{f1_train:.5f},\\t测试集上的F1得分：{f1_test:.5f}\")\n",
    "    print(F\"训练集上ROC面积: {roc_area_train:.5f}, 测试集上ROC面积: {roc_area_test:.5f}\")\n",
    "\n",
    "    _, ax = plt.subplots(figsize=(8,8))\n",
    "    plot_roc_curve(clf, X_train, y_train, label='ROC on train', ax=ax)\n",
    "    plot_roc_curve(clf, X_test, y_test, label='ROC on test', ax=ax)\n",
    "    ax.legend(fontsize=12)\n",
    "    plt.show()\n",
    "\n",
    "estimate_model(logis)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 基于 **`SGD`** 算法分类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.2.1 SGD 模型的理论算法\n",
    "\n",
    "&emsp;&emsp;在深度学习里，目标函数通常是训练数据集中有关各个样本的损失函数的平均。设 *$f_{i}(x)$* 是有关索引为 $i$ 的训练数据样本的损失函数, $n$ 是训练数据样本数, $x$ 是模型的参数向量，那么目标函数定义为：\n",
    "\n",
    "$$f(x) = \\frac{1}{n} \\sum_{i=1}^{n} f_{i}(x)$$\n",
    "\n",
    "&emsp;&emsp;目标函数在 $x$ 处的梯度计算为：\n",
    "\n",
    "$$\\nabla f(x) = \\frac{1}{n} \\sum_{i=1}^{n} \\nabla f_{i}(x)$$\n",
    "\n",
    "&emsp;&emsp;如果使用梯度下降，每次自变量迭代的计算开销为 $O$ ($n$),它随着$n$线性增长。因此，当训练数据样本数很大时，梯度下降每次迭代的计算开销很高。\n",
    "\n",
    "&emsp;&emsp;随机梯度下降（stochastic gradient descent，SGD）`减少了每次迭代的计算开销`。在随机梯度下降的每次迭代中，我们随机均匀采样的一个样本索引$i$ ∈ { 1 , … , n } ,并计算梯度 $\\nabla f_{i}(x)$ 来迭代 $x$ :\n",
    "\n",
    "$$x \\leftarrow x - \\eta \\nabla f_{i}(x)$$\n",
    "\n",
    "&emsp;&emsp;这里 $\\eta$ 同样是学习率。可以看到每次迭代的计算开销从梯度下降的 $O(n)$ 降到了常数 $O(1)$。值得强调的是，随机梯度 $\\nabla f_{i}(x)$ 是对梯度 $\\nabla f(x)$ 的`无偏估计`：\n",
    "\n",
    "$$E_{i} \\nabla f_{i}(x) = \\frac{1}{n}  \\sum_{i=1}^{n} \\nabla f_{i}(x) = \\nabla f(x)$$\n",
    "\n",
    "&emsp;&emsp;这意味着，平均来说，`随机梯度是对梯度的一个良好的估计`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.2.2 基于 **`网格搜索`** 的 **`SGD`** 分类模型训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipe = Pipeline([\n",
    "    (\"tfid\", TfidfVectorizer()),\n",
    "    (\"sgd\", SGDClassifier(loss = \"log\"))\n",
    "])\n",
    "\n",
    "param_grid = [{\n",
    "    \"tfid__ngram_range\": [(1, 1)],\n",
    "    \"tfid__tokenizer\": [tokenizer],\n",
    "    \"sgd__penalty\": ['l1'],\n",
    "    \"sgd__alpha\": [0.00001]\n",
    "}]\n",
    "\n",
    "sgd_clf = GridSearchCV(pipe, param_grid, n_jobs=-1)\n",
    "\n",
    "sgd_param = sgd_clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.2.3 **SGD** 分类模型的各项评分及 ROC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练集上的准确率：0.94129,\t测试集上的准确率：0.93623\n",
      "训练集上的精度：  0.91721,\t测试集上的精度：0.91314\n",
      "训练集上的召回率：0.96993,\t测试集上的召回率：0.96377\n",
      "训练集上的F1得分：0.94284,\t测试集上的F1得分：0.93777\n",
      "训练集上ROC面积: 0.98175, 测试集上ROC面积: 0.97897\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAHgCAYAAAC1uFRDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABJW0lEQVR4nO3deZgdZZn38e99ll7SS/aVbCwJiwugIYCorLILOCDKCEwcR0cHRngZdXwFBRlHHZcZBzdcRsEFEFARkOUFR1BMEEH2EDCyZCVLZ+v9LHW/f1R1p9N0uk8nfU515fw+13WurqpT59Td1Un/+ql6zvOYuyMiIiLJk4q7ABEREdk1CnEREZGEUoiLiIgklEJcREQkoRTiIiIiCaUQFxERSahM3AUM16RJk3zu3LlxlyEiIlIRjz322EZ3nzzQc4kL8blz5/Loo4/GXYaIiEhFmNkrO3tOl9NFREQSSiEuIiKSUApxERGRhFKIi4iIJJRCXEREJKEU4iIiIgmlEBcREUkohbiIiEhCKcRFREQSSiEuIiKSUApxERGRhFKIi4iIJJRCXEREJKEU4iIiIglVthA3sx+Y2Xoze2Ynz5uZXWNmy83sKTN7U7lqERER2ROVsyV+HXDyIM+fAsyLHh8Cvl3GWkRERPY4mXK9sbv/zszmDrLLmcCP3N2Bh81snJlNd/e15apJRERkpLg7+aITBAFeLFAs5gmKeTwoMnb85IrUULYQL8FewMo+66uibQpxEZE+3B13CNwp9iwHQfQoEOS6KBYLBMUC7gEUw+14kaAYYF7EgyJe6IKgCFHQEERfi3ms2IUXAyDAg4BCMSAIigRBQDFabu/OU5s23APcHTyIHo57QH3nq3Rlx0FQxAjCWjyAoIgHfdY9wDxgTPdGCqkaAlLRtmLvV/OA7lyOujSkPE8qKISvxXtfD44HAUZYg/W8d88+BGS8SIpitM0xwgc4Blj0vRhOysAIsPCsYx7tE23reW3PI4WToUidFXf4eW2lAa5aU5F/G3GGeMnM7EOEl9yZPXt2zNWIyA6CAIq56JGHfHsYFL2/3MMgKBQKFIMwDLxYpBiEYeFBEe9upUgKD5xCsYAREBTDX/pBUCTMi2IYHkERD8L39SDAOjaQS9WDBwQ924vhvrl8+Is/EwWKe0Cm0EE230ZXegwWBBC9jwdhAIa/1IsQBFjnJurSTrfV9QZVGB5h7eaOs2N4mPcJFQIyXiBFQIrw+VT0fAqPtm1frvE8k2wrW70BoPc16WgfIyCNk7Eg3p/5Lgjcou/Wer9zw6mzPBu8OTpDKYq9ZytNHUbBU1gqTd6yBKTD11kUoZYCS4Flti+nUmAWbQ/PWDGVCY8WbaP3PaL9gO6ik81kSKeiu8z99qX3kQK2L7fnjTH1dXgqi6cyYGmoGcPhFTqvcYb4amBWn/WZ0bbXcPfvAt8FWLBggZe/NJFRyJ1iIU9XZxu5znY6OzvId7ZS6Gonl8+Tz+UICjmKhW7SHRvoooag0E2uO0dXVycTixvoSo3BijksyFOb34Z3boaaRlJewLxAKgi/TsmtZFt6PEEhT4YiWQqkPU+Nd5OJWkVZ8jTSMWTZBqSjx2jT6TV9QiOKU9u+DsZY38am1ATyVtP7S98tjNQdwiS1fdlI4dEv+cDSBJah2Pu67eGzw3L0WBd0A0ZHdhxYCrMUpNJRqETv3xtY4fa6/Fba66bSVYAxdbXb90ulCSzdu7+ns3i6Fk+lIZXBLQOpTPg+qQxBqgZLhe+fzaRJp9Jk0ilS6fCrY9Rks6RTBtF+qeg4ZkYqncIydWApUqk0lkqTSoXL6UyKbCrVm4UZM1JmuMFks5j/JSRXnCF+O3Cxmd0EHA5s1f1wiUvPva3uQpHOXJHuQkBXrkBXrkC+0E0+l6dQyLO5tYPaVJFidydb2tp7w41CDit0YoUuUsUuGjvXkPM04ztfodPqsSCPBQUI8qSCHJliN2nPkwrCx6ziSjbbWNJewIt5alMBM9hAB7WYh5fsslYkDTREj/G78f3mPGzb1VmebW31tFkjRctQJE2RNB0Gzfm1rEtPZUsxQzrTjKczFFI15FN1FC1LeyHFmJo0NamAbTVTIJXFU1nq6KY7O44gU0PKtv8it1SK1u6AcQ210S/5MITMjJSlSZOnkG0GS9FddBrrakmlDEulovcxSKW3h0YqRSrVEyppUtk6UunweOlUilQ60xsgmUyadDoT7pdOkU6HNWVSRq0ZqdTgITJjN861SDmVLcTN7EbgGGCSma0CrgSyAO5+LXAXcCqwHOgA3l+uWiS53J3uQkAu102xq51Cx2aK+S5yXZ3kuzvxjk205QJSQZ5cdzf5rjZSXZugkKO1vQMvdFNTbCftOaxYoKO7m/q0Y0Ge6flVtDCWdJAj7Xka6WSMdVNDnnEUqCVHTb97XbuqlTEUyVC0DAXLULAsOaulmKqhmM6yPjOHBm9jY80ciqTJkWFNtpaG4la21M3E01lIZWgvGOPrUpCpJxgziXRNPbVWwBumkK2tJZ2pIZ3Jkk0Z6TFjqa2tJ1tTR01tDTXZWlK1DZCuoSa1/YMpzdFjINNH5LsXkXIpZ+/084Z43oGLynV8iU9P8G7tzLNt2za62rdQbN1Acetq8p3tdHR1kevuJLttJZ7rYGLXK+S7O0injHyum7pUkaznaaKderpoopNmy+9SLQVP0Ukt3ak6AjIULU2hkCKVyUI2wzxfx+a6WeElxkwt27LNdJGlvq4ey9Ri2XrS2RpS6SypTAZSWTLZLDUpyGQypBomks7Wks7WkcrWka5tJFVTD5kaqJ8ANY2QztBUYr2zht5FRKRXIjq2SWW5Ox25Imu3dtLSlmNzR472zm7Wb1jPuGAz6ba11LatorlzJV35AhM7V9DmtWSCbuYEK9ka1LO/rWSq5Zla4jG7rI5Vvhc0NpELUmRr62jPjGFLppnV2XoKNc0EmTHUkKNrzEzSNXWks7VkU46PmYSns4xtbKS+vp7apklk6xqoqa0jm83SBIOG6LjdP2UiIrFQiFeJIHA2tnezfH0bm9vztHbl2dKRI791HRvWvsy4wgYKuS4mt/+F5vwGxngHs2wDe9sWDqadukFawt1WR8GybM1OIUiPY7rnWNl4NDUpKDZMIT9+HlY3lnTzVOrrxzBm3BQaGxpIZ+ugfjxk66gD9qvc6RAR2SMoxPcA7s6WjjyrNnfy9OqtvNzSzubWDgptLUzd8memti1jQmE9RsB4Wjki9QoBxmTbNvAbpiCfqiNfM45Cw1zaJsyndewsxtTXU9swlnTTFGzC3jBuDtSPoxaoJexs1aMywxyIiFQ3hXiC5AoBKza1s+zVVpatbWXZq62s2tzB5i2b2D+3lENtOQtSz/P21KvMsBZS9Pk0Xho6ayaSa5yJNxxOAzlyM95ItqYOm3Eo1DaF92+bZ0DjVLKpVNgLUURERi2F+ChVDJwVmzr400ub+POKzTy1aivPr2slE3Szn63mxPRjXFi3ln1tDdNZQaomDOwgVYPvcwyp6W+EhskwYW+YeRiMmUA9UB/vtyUiIiNIIT6KrGjp4K5n1vLwiy089vJmWrsLAEyogzMmrOTq2X/m9Zvupy7XEr6gAMw7EaafC3u9GSbNJzVhn2iUIRER2dMpxGPW0tbNd373Ig/9ZSNL14b3qPeZ3MDpB8/gLRPbeNva6xi37CbYBKRrYO+j4Q3vhulvhMkHKLBFRKqYQjwmuULA9Ytf5r9/8xfaugvMmlDPv558AKe8fhpzG4vwuy/BA98CL8LEeXDwe+DwD4f3rkVERFCIx+Jnf1rBV/7fC2xo7Wbh3hP43FmvZ/7UJijk4KH/ggc+H+647/Fw2lfD+9oiIiL9KMQraENrN//+66Xc9sQaDpjWxBf/5g0cd8CUcNq7J2+Cez8FHS0waX845pPwunfpcrmIiOyUQrxCFi/fyMU3Ps6WjhwffNvefOLkA8imU7B1FXz/HdC6Jhz45KQvwBEfUXiLiMiQFOIVcOtjq/jYLU8yc3w9P/r7t/L6vcaGTyy/H255P+Q74W0fC1vfaX06W0RESqMQL7NfPr6KT9z6JG+eM54f/N1hjB0ThfSKh+EnZ4fLH34Ipr0hviJFRCSRUkPvIrvqBw+9xP/52ZMcNKOZ7124YHuAP38P/PhvoG4sfOgBBbiIiOwShXiZ/HjJy1x951KOnj+Zmz50JBMaasInHr4WbnwPeACLfg0zDo23UBERSSxdTi+D+5au4zO3P8sR+0zg+3+3IOzABrBtLdzzrzBpfhjgjVPiLVRERBJNLfERtmpzB//686eYNX4MP1y0cHuAA/z8H8KvZ12rABcRkd2mlvgIcnf+9edP0Z0v8qO/X0h9TXr7k4//BF55CI6/Ema+Ob4iRURkj6GW+Ai69sEX+cPyFi47cf/tHyMDaF0Hv7oIxu8NR/xTfAWKiMgeRSE+Qta3dvH1//0LJxw4hb8/au6OT95xSfj1lC9Btq7itYmIyJ5JIT5Cvnb/X+guBPzfUw/E+o629pf74YW74S3/DPNPjK9AERHZ4yjER8BfN7Rx4yMrOG/hLPad3Lj9CXe479OQysJbL4uvQBER2SMpxEfADX9cAcBFx+634xP3Xg7rl4bDqY6ZEENlIiKyJ1OI76b1rV3c/OhKjp4/melj67c/0d4CD38TGqaoFS4iImWhEN9NX77neTpyRT5x0gE7PvG7L4df3/VtSOk0i4jIyFO67IZ127r41RNreM9hszhoRvP2J1Y9Co/+D0x5Hex3QnwFiojIHk0hvht+8vAr5IoB//DWvXd84s7/A7XN8Hd3xFOYiIhUBYX4LsoXA677w8ss3HsC+/Ttkb5+Gbz6FBxyHjRMjK9AERHZ4ynEd9F9S9fR2l3ggiPm7PjE4q+HXw/7h8oXJSIiVUUhvotufGQFkxprOfUN07dv7NwMS2+DvY+G8XPjKk1ERKqEQnwXbO3I89DyjZy7YCbpVJ/R2R7+NuTaws+Fi4iIlJlCfBc88vIm3OHo+ZO3b3SHJd+C2UfCnLfEV5yIiFQNhfgueOSlFmoyKQ6eNW77xk0vQq4VZh4WW10iIlJdFOK74I8vbeKQWeOoy/aZL/ypm8Ovb14US00iIlJ9FOLD1NZd4JnVWzli735joS/7Ncx4E0zcN57CRESk6ijEh+mxVzYTOCzcu89nwNc+BeuehoPPi68wERGpOgrxYfrjiy1kUsab5ozbvvHJG8PpRt9wTmx1iYhI9VGID9MjL23iDTPHMqYmE24o5sP74fufoulGRUSkohTiw9CZK/Lkqi0s7Hs//C/3QcdGOOR98RUmIiJVSSE+DI+v3Ey+6BzR9374Ez8N5wzf7/j4ChMRkaqkEB+GP764iZTBm+eODze0b4QX7oE3ngvpbLzFiYhI1VGID8MjL23ioBnNNNdFgf30rRAU4JC/jbcwERGpSgrxEm1qz/HYK5s5vP+l9OmHwNTXxVaXiIhUL4V4ie5b+iq5YsCZh8wIN7z6dDRvuDq0iYhIPBTiJXpoeQuTGmt5w15jww1P6LPhIiISL4V4CdydJX9t4Yh9JmBm0WfDfwb7n6zPhouISGwU4iX464Y2NrZ185Z9J4Ub9NlwEREZBRTiJfjD8hYA3jYvCvHffBbStbDvcTFWJSIi1U4hXoJHXtrE5KZaZo6vh0I3bFgG9eMhUxt3aSIiUsUU4kPIFwN+/5cNvH3e5PB++Mu/D5844sPxFiYiIlVPIT6Ev6xrY1tXgbfPjy6lP3dH+PXN74+vKBERERTiQ3p+3TYA5k9tCjf89bewz7FQPy6+okRERFCID+mZ1duoSafYd3IjbHoRtryiyU5ERGRUUIgPYdmr2zhgehM1mRQ8d2e4cZ9j4y1KREQEhfiQVm/uDHulAzx5I6QyMHn/eIsSERFBIT6k9a3dTGuuB3dYvxQOPk/TjoqIyKigEB/E1s48HbkiU5prYcPz4cYJe8dblIiISEQhPohXWtoBmDNhTDjtKMAB74yxIhERke0U4oN4uaUDgL0nN4TTjgJMnh9jRSIiItspxAfx51c2AzBz/BhY9yxMPzjmikRERLZTiA9iS0eOTMpo7FgF7RtgzlvjLklERKSXQnwQL21sZ8Hc8fDS78INB50Zb0EiIiJ9KMR3IgicJ1dt5YBpzfDX/w1nLZt5WNxliYiI9FKI78RLUc/0WRPGwJrHYfaRkNLpEhGR0UOptBNrtnQCcOiEPGx+GaYfEms9IiIi/SnEd2Lt1i4ApudXhhvGzY6xGhERkddSiO/Eqk3hZ8Qnb3ki3DDnyPiKERERGYBCfCde2dTBpMZaMiv+EG4YNyfegkRERPpRiO/EyxvbGVufgUI3WArM4i5JRERkBwrxnejIFZnWVAOvPASH/G3c5YiIiLxGWUPczE42s+fNbLmZfXKA52eb2W/N7HEze8rMTi1nPaXKFwP+sr6NYydvDTeM18xlIiIy+pQtxM0sDXwTOAU4CDjPzA7qt9sVwM3ufijwXuBb5apnOJavbwNgjq8NN+z1phirERERGVg5W+ILgeXu/qK754CbgP7jljrQHC2PBdaUsZ6SrdocfkZ8fmp1uGGGQlxEREafTBnfey9gZZ/1VcDh/fa5Cvh/ZvbPQANwQhnrKdkzq8PL6FO7XoTGqVA/Lt6CREREBhB3x7bzgOvcfSZwKvBjM3tNTWb2ITN71Mwe3bBhQ9mLcncAajc8DXVjy348ERGRXVHOEF8NzOqzPjPa1tcHgJsB3H0JUAdM6v9G7v5dd1/g7gsmT55cpnK329CWY2JDDda9DdI1ZT+eiIjIrihniP8JmGdme5tZDWHHtdv77bMCOB7AzA4kDPHyN7WHsKm9m4mNNVDogon7xl2OiIjIgMoW4u5eAC4G7gWeI+yF/qyZXW1mZ0S7/QvwQTN7ErgRWOQ917JjtHpLJ3Mbi9C1Faa+Pu5yREREBlTOjm24+13AXf22fabP8lLgqHLWsCu2dub5m7qnw5VJ8+MtRkREZCfi7tg2Km1pz7NP8Eq4os+Ii4jIKKUQ76e7UKS1u8BYC2cxY+yswV8gIiISE4V4P+u3dQMwo/uv4QZNfCIiIqOUQryf9a1hiGczacg2xFyNiIjIzinE+1m7NRxydUz7Ktjv+JirERER2TmFeD8tbTka6aCubSVMfV3c5YiIiOyUQryfZa+2cnAquh8+bk68xYiIiAxCId5PNm1MJppHfNK8eIsREREZhEK8nxfWtTJtTLTS8Jph3EVEREYNhXg/m9pzTPKN4Yp6p4uIyCimEO8nZUaqJmqK1yjERURk9FKI99PaVWBGQzTAS6Y23mJEREQGoRDvZ1tnnr2KK8OVVDreYkRERAahEO/D3WntLjCxsC7uUkRERIakEO+jKx8A0O1ZqB8fczUiIiKDK+t84knTXSgC0JAuwNRDY65GRERkcGqJ95ErhC3xce0vQVqd2kREZHRTiPfRngtb4vlMI3S3xlyNiIjI4BTifXTli4DT0LVWQ66KiMiopxDvo727QBPhVKRkxwy+s4iISMwU4n1s68oz1trDlcYp8RYjIiIyBIV4HytaOhhLW7gybla8xYiIiAxBId5HOp2ilny44h5vMSIiIkNQiPfRnS9Sb93hStP0eIsREREZgkK8j5WbOmikK1zJ1sdbjIiIyBAU4n2Mqc1QSy5cqW2OtxgREZEhlDTsqpmlgIOBGUAn8Iy7ry9nYXF4paWdfWu2hiuahlREREa5QUPczPYF/hU4AfgLsAGoA+abWQfwHeB6dw/KXWgl1KRTNBS2hmeltjHuckRERAY1VEv8c8C3gX9037G7tplNAf4WuAC4vjzlVVa+6EyrL0IeqBsXdzkiIiKDGjTE3f28QZ5bD3xtpAuKU2e+SF2qAGMmglnc5YiIiAxqlzu2mdk7RrKQ0aCtu8A03whNM+IuRUREZEi70zv9f0asilGiK19kdnElNE6OuxQREZEhDdWx7fadPQVMHPly4tWRK1JMZaBrW9yliIiIDGmojm1vA86HngHFexmwsCwVxai9u8Dk/FqYusfdKRARkT3QUCH+MNDh7g/2f8LMni9PSfHpyhXChUJ3vIWIiIiUYKje6acM8tzbR76ceNUUWiGNpiEVEZFE0LCrkSBwLN8RrkzYN95iRERESqAQj+SKAWOtPVzRkKsiIpIACvFIdyGgmaglnq6JtxgREZESKMQj3fki81OrwhXNJS4iIglQcoib2VWDrSdddyGgkc5wZcI+8RYjIiJSguG0xB8bYj3ROvNFptmmcKVhUrzFiIiIlKDkEHf3OwZbT7pN7Tm6iO6Fp7PxFiMiIlKCoYZd/TrgO3ve3T864hXFJAicDEUKmTFDjoAjIiIyGgyVV49WpIpRoLsYkKEIKUW4iIgkw1Ajtl3fd93Mxrh7R3lLiseGbd3UkoOULqWLiEgylHRP3MyONLOlwLJo/WAz+1ZZK6uwmkyK+anVBLVj4y5FRESkJKV2bPsacBLQAuDuTwJ71NjpuWJAioBUoMlPREQkGYbTO31lv03FEa4lVvkoxIsT5sddioiISElK7cW10szeAriZZYFLgOfKV1bl5QsBWYpYVuOmi4hIMpTaEv8wcBGwF7AGOCRa32N05gNm2XpSGY2bLiIiyVBSS9zdNwLvK3MtscoVArqpoamzJe5SRERESlJq7/R9zOwOM9tgZuvN7FdmtkcNMB7eE3ds8v5xlyIiIlKSUi+n3wDcDEwHZgC3ADeWq6g45IoBE20bZOriLkVERKQkpYb4GHf/sbsXosdPgD0q7VZsbA0XulvjLURERKREQ42dPiFavNvMPgncRDiW+nuAu8pcW0VNqQvChUn6iJmIiCTDUB3bHiMMbYvW/7HPcw7833IUFYd0LmqB59rjLURERKREQ42dvnelColbvlgIF8buFW8hIiIiJSp5yi4zez1wEH3uhbv7j8pRVBzWbm4LFywdbyEiIiIlKinEzexK4BjCEL8LOAV4CNhjQnx8XRTemopUREQSotTe6ecAxwOvuvv7gYOBPWq6r2LP5fSUWuIiIpIMpYZ4p7sHQMHMmoH1wKzylVV5uVw+XFCIi4hIQpR67fhRMxsHfI+wx3obsKRcRcWho21LuOAeax0iIiKlKnXs9H+KFq81s3uAZnd/qnxlVd7Y2hR0AOls3KWIiIiUZKjBXt402HPu/ueRLykeac+FC/Xj4y1ERESkREO1xL86yHMOHDeCtcRqXLA5XEhrPnEREUmGoQZ7ObZShcStEESD0ulyuoiIJESpvdP3eJney+njYq1DRESkVArxyLRgXbigqUhFRCQhFOKR9d3RZfSahngLERERKVFJIW6h883sM9H6bDNbWMLrTjaz581seTSV6UD7nGtmS83sWTO7YXjlj5y6dPT5cA27KiIiCVFqYn0LCAh7o18NtAI/Bw7b2QvMLA18E3gHsAr4k5nd7u5L++wzj3A606PcfbOZTdml72IEpL0YLqTUsU1ERJKh1Mvph7v7RUAXgLtvBmqGeM1CYLm7v+juOeAm4Mx++3wQ+Gb0frj7+pIrH2EpNHa6iIgkS6khno9a1g5gZpMJW+aD2QtY2Wd9VbStr/nAfDP7g5k9bGYnl1jPiAoCZybR3w9mcZQgIiIybKVeTr8G+CUwxcz+nXBWsytG6PjzCKc5nQn8zsze4O5b+u5kZh8CPgQwe/bsETjsjvJBwGZvGvH3FRERKadSx07/qZk9RjgdqQFnuftzQ7xsNTvOdDYz2tbXKuCP7p4HXjKzFwhD/U/9jv9d4LsACxYsGPEZSvJFJ02RznQT9SP95iIiImVSau/0a4AJ7v5Nd/9GCQEOYRDPM7O9zawGeC9we799biNshWNmkwgvr79YYu0jplh00gTqmS4iIolS6j3xx4ArzOyvZvYVM1sw1AvcvQBcDNwLPAfc7O7PmtnVZnZGtNu9QIuZLQV+C3zc3VuG/23snkIQkKaImzq1iYhIcpR6Of164HozmwCcDfyHmc1293lDvO4u4K5+2z7TZ9mBy6JHbIrupHFcPdNFRCRBhjti237AAcAcYNnIlxOPYuDMtA1oADsREUmSUu+Jf8nM/kI40MszwAJ3f2dZK6ugQtFpo57a3Oa4SxERESlZqT25/goc6e4by1lMXIqBk6VAa/O+jI+7GBERkRINGuJmdoC7LyPsaT7bzHb4kLa7/7mcxVVKZ75Ig3WRT42JuxQREZGSDdUSv4xwkJWvDvCcE46lnnjuUEcOz0yKuxQREZGSDRri7v6haPEUd+/q+5yZ7TETb+eLATUUSGVr4y5FRESkZKV2x15c4rZEyhcDxlkbpIea00VERGT0GOqe+DTCSUvqzexQwiFXAZqBPeYG8tbOPAfRTnexI+5SRERESjbUPfGTgEWE457/Z5/trcCnylRTLDqpIahV33QREUmOoe6J94zUdra7/7xCNVVcvhiQoUihXiEuIiLJMdTl9PPd/SfAXDN7zdCo7v6fA7wscfJFZ6x1sDmre+IiIpIcQ11Ob4i+Npa7kDjlC0UAMt1b4i1ERERkGIa6nP6d6OtnK1NOPDZuawcgGL9PzJWIiIiUbjhjpzebWdbMfmNmG8zs/HIXVymNmQCAVDobcyUiIiKlK/Vz4ie6+zbgdOBlwtnMPl6uoiqtWMgDkNFgLyIikiClhnjPZffTgFvcfWuZ6olHvhOAlDq2iYhIgpQ6i9mdZrYM6AQ+YmaTga4hXpMYQT78VtI1e8z4NSIiUgVKaom7+yeBtxDOI54H2oEzy1lYJQXF8HJ6OqN74iIikhwltcTNLAucD7zdzAAeBK4tY10Vlc+HIW7q2CYiIglS6uX0bwNZ4FvR+gXRtn8oR1EVF7XESZV6OkREROJXamod5u4H91n/XzN7shwFxSFd6IwW1LFNRESSo9Te6UUz27dnxcz2AYrlKSkGhaiPXu0ePTCdiIjsYUptiX8c+K2ZvUg4Hekc4P1lq6rCPIj+HtHldBERSZAhUyv6ONlWYCEwJdr8vLt3l7OwStrSHl1OV4iLiEiCDHo53cz+AXgW+DrwBDDX3Z/akwIcYExPdqfSsdYhIiIyHEM1PS8FXufuG6L74D8Fbi97VRWW8uhyuinERUQkOYbq2JZz9w0A7v4isEcOLt5Q2BQu6HK6iIgkyFCpNdPMrtnZurt/tDxlVVbOoxa4BnsREZEEGSrE+89U9li5ConTmMK2cKFubLyFiIiIDMOgIe7u11eqkDgVOzaHC1lNgCIiIskxVO/075nZ63fyXIOZ/b2Zva88pVVOuia61a8QFxGRBBnqcvo3gc+Y2RuAZ4ANQB0wD2gGfkDYYz3RMl6gQJpMqtQB7EREROI31OX0J4BzzawRWABMJ5xT/Dl3f7785VXGlMJavOQRaEVEREaHkj5T5e5twAPlLSU+bYwhSz7uMkRERIZFzU8AnK3p8XEXISIiMiwKcQAPcDRam4iIJMuwQtzM9szu20GAm8VdhYiIyLCUFOJm9hYzWwosi9YPNrNvlbWySvIAN12UEBGRZCk1uf4LOAloAXD3J4G3l6uoSjMC9U4XEZHEKTm53H1lv03FEa4lNuYOaomLiEjClDpt10ozewvgZpYFLgGeK19ZlRW2xHVPXEREkqXU5ueHgYuAvYDVwCHAP5WppopLoXviIiKSPKW2xPd39x3GSDezo4A/jHxJldfg7QpxERFJnFKT6+slbkukFAFp32Nu8YuISJUYtCVuZkcCbwEmm9llfZ5qhj1ndBRzpz07nolxFyIiIjIMQ11OrwEao/2a+mzfBpxTrqIqLbwnXuqdBRERkdFhqFnMHgQeNLPr3P2VCtVUcSmKuO0xFxZERKRKlNr87DCzLwOvI5xPHAB3P64sVVVYygNQiIuISMKU2rHtp4RDru4NfBZ4GfhTmWqqKHcnRUCgEdtERCRhSk2uie7+P0De3R90978H9ohWuDsKcRERSaRSkysffV1rZqeZ2aHAhDLVVFEOzLVXSacV4iIikiyl3hP/nJmNBf6F8PPhzcCl5SqqkgJ31vt4GvOb4y5FRERkWEoKcXe/M1rcChwLvSO2JZ47GM62uhlMi7sYERGRYRhqsJc0cC7hmOn3uPszZnY68CmgHji0/CWWl+MYmsVMRESSZ6iW+P8As4BHgGvMbA2wAPiku99W5toqIuzY5gxjVlYREZFRYagQXwC80d0DM6sDXgX2dfeW8pdWGT290zUBioiIJM1QyZVz9wDA3buAF/ekAIewY1vKdDldRESSZ6iW+AFm9lS0bMC+0boB7u5vLGt1FeCge+IiIpJIQ4X4gRWpIkaBe3hP3CzuUkRERIZlqAlQ9thJT3ronriIiCRV1SeX97TEdSpERCRhqj65elriupwuIiJJU3KIm1m9me1fzmLiELiTpaCObSIikjglJZeZvRN4ArgnWj/EzG4vY10VE15Id4JUNu5SREREhqXU5udVwEJgC4C7P0E4t3ji9YzYpo5tIiKSNCVPReruW/tt85EuJg7uGjtdRESSqdSpSJ81s78F0mY2D/gosLh8ZVVOz+V0hbiIiCRNqcn1z8DrgG7gBsIpSS8tU00VVQwcI8BQ73QREUmWUlviB7j75cDl5SwmDoWiY0BnYY+4OyAiIlWk1Jb4V83sOTP7NzN7fVkrikGKgDG1pf49IyIiMjqUFOLufixwLLAB+I6ZPW1mVwz1OjM72cyeN7PlZvbJQfY728zczBaUXPkIKXrYEjfdExcRkYQpObnc/VV3vwb4MOFnxj8z2P5mlga+CZwCHAScZ2YHDbBfE3AJ8MfSyx45QRBoKlIREUmkUgd7OdDMrjKzp4GvE/ZMnznEyxYCy939RXfPATcBZw6w378B/wF0lV72yAmCAFBLXEREkqfU5PoB4UAvJ7n7Me7+bXdfP8Rr9gJW9llfFW3rZWZvAma5+69LrGPEBUHUoU0hLiIiCVNSby53P3KkD2xh0/c/gUUl7Psh4EMAs2fPHtE6ikExPEZKHzETEZFkGbT5aWY3R1+fNrOn+jyeNrOnhnjv1cCsPuszo209moDXAw+Y2cvAEcDtA3Vuc/fvuvsCd18wefLkob+rYfDocrpa4iIikjRDtcQvib6evgvv/SdgnpntTRje7wX+tufJaBjXST3rZvYA8DF3f3QXjrXLgqglntJUpCIikjCDNj/dfW20+E/u/krfB/BPQ7y2AFwM3As8B9zs7s+a2dVmdsZIFD8iCt0AZPP9h4YXEREZ3Uod4eQdwL/223bKANt24O53AXf12zbgR9Pc/ZgSaxlRHV1hiOcaZsRxeBERkV02aIib2UcIW9z79LsH3gT8oZyFVUrGwnviuUCX00VEJFmGaonfANwNfAHoO+Jaq7tvKltVFdTTsa2upibmSkRERIZnqBB3d3/ZzC7q/4SZTdgTgtyDAgCWSsdciYiIyPCU0hI/HXiMcOrtvtecHdinTHVVjPd+TlwhLiIiyTJoiLv76dHXvStTTuUFxbAljkJcREQSptSx048ys4Zo+Xwz+08zG9mh02KSzrcDkC20x1yJiIjI8JQ6TNm3gQ4zOxj4F+CvwI/LVlUFFaOh0wtjpsRbiIiIyDCVGuIFd3fCWci+4e7fJPyYWeJtvyde6kfmRURERodSk6vVzP4vcAHwtmjykmz5yqocdWwTEZGkKrUl/h6gG/h7d3+VcDKTL5etqkrqGTs9rRAXEZFkKSnEo+D+KTDWzE4Hutz9R2WtrEK2dYTDrqolLiIiSVNq7/RzgUeAdwPnAn80s3PKWViljKkJP/rumopUREQSptR74pcDh7n7egAzmwzcD9xarsIqJhqxLZNRxzYREUmWUpufqZ4Aj7QM47WjWrY7nII0HeRjrkRERGR4Sm1+3mNm9wI3Ruvvod8Uo0mVT9UC4HVjY65ERERkeEoKcXf/uJn9DfDWaNN33f2X5Surgjycxcx0T1xERBJmqPnE5wFfAfYFngY+5u6rK1FYpYRj2ICZ5hMXEZFkGar5+QPgTuBswpnMvl72iiqtJ8T1ETMREUmYoS6nN7n796Ll583sz+UuqNK893K6WuIiIpIsQ4V4nZkdyvZ5xOv7rrt78kO9J8RTCnEREUmWoUJ8LfCffdZf7bPuwHHlKKqSeu6Jp0yX00VEJFkGDXF3P7ZShcTF1RIXEZGEqvrPVW3vna6WuIiIJEvVh7ipJS4iIglV9SGuz4mLiEhSlTqLmZnZ+Wb2mWh9tpktLG9plbG1IwdoxDYREUmeUpPrW8CRwHnReivwzbJUVGFNteEpyKQV4iIikiylToByuLu/ycweB3D3zWZWU8a6Kmb75XSFuIiIJEupyZW3sPu2Q+984kHZqqokhbiIiCRUqcl1DfBLYIqZ/TvwEPD5slVVQbX5bQCkUgpxERFJllKnIv2pmT0GHE845OpZ7v5cWSurkGLUAtdHzEREJGlKCnEzmw10AHf03ebuK8pVWKUE0cUIy9TFXImIiMjwlNqx7deE98MNqAP2Bp4HXlemuiqmp2MbuicuIiIJU+rl9Df0XTezNwH/VJaKKs17+ufpcrqIiCTLLjU/oylIDx/hWmLR0xBXS1xERJKm1Hvil/VZTQFvAtaUpaIKMy9GC2qJi4hIspR6T7ypz3KB8B75z0e+nMrbfk9cIS4iIskyZIhHg7w0ufvHKlBP5emeuIiIJNSgN4LNLOPuReCoCtUTH7XERUQkYYZqiT9CeP/7CTO7HbgFaO950t1/UcbaKqKtqztcUMc2ERFJmFLvidcBLcBxbP+8uAOJD/G6TE94qyUuIiLJMlSIT4l6pj/D9vDu4QO/JFkMDfYiIiLJNFSIp4FGBm6m7hEh7kHUsU33xEVEJGGGCvG17n51RSqJiaslLiIiCTVUcu35zdNAHzETEZFkGirEj69IFTEam18XLqglLiIiCTNocrn7pkoVEpdt6XHhQrrUjvoiIiKjQ9U3P9s6cxRIx12GiIjIsFV9iDdkjUD3w0VEJIGqPsQNx9USFxGRBKr6EMcDtcRFRCSRqj7EzQNcA72IiEgCVX2I4wGu0yAiIgmk9CLAdTldREQSqOpDPOUBgQZ6ERGRBKr69Bpb3Kx2uIiIJFLVh3iBNE3BtrjLEBERGbaqD/Gc1dCSnhR3GSIiIsNW9SFuHhAMOSOriIjI6FP1IQ6uz4mLiEgiVX2Id3bn9REzERFJpKoP8fpsikLgcZchIiIybFUf4oaTTmsCFBERSZ6qD3Fw0OV0ERFJoKoPcXPHNWKbiIgkUNWnlxHEXYKIiMguqfoQx9EsZiIikkhVn16mWcxERCShqj7EwUGDvYiISAJVfYgbqCUuIiKJVNYQN7OTzex5M1tuZp8c4PnLzGypmT1lZr8xsznlrGfAGj1AHzETEZEkKluIm1ka+CZwCnAQcJ6ZHdRvt8eBBe7+RuBW4EvlqmdnTGOni4hIQpWzJb4QWO7uL7p7DrgJOLPvDu7+W3fviFYfBmaWsZ4BzfQ1pL1Y6cOKiIjstnKG+F7Ayj7rq6JtO/MB4O4y1jOgDUwk692VPqyIiMhuGxUTaZvZ+cAC4OidPP8h4EMAs2fPHtFjZyiwNTuFySP6riIiIuVXzpb4amBWn/WZ0bYdmNkJwOXAGe4DN4nd/bvuvsDdF0yePLJxm6JIYKPibxkREZFhKWeI/wmYZ2Z7m1kN8F7g9r47mNmhwHcIA3x9GWvZqYwrxEVEJJnKFuLuXgAuBu4FngNudvdnzexqMzsj2u3LQCNwi5k9YWa37+TtymYOa/QJMxERSaSyNkHd/S7grn7bPtNn+YRyHr8UGxlHXbEt7jJERESGrepHbPMgYEt2WtxliIiIDFvVh3janC59TFxERBKo6kM8hVNXk427DBERkWFTiBNo2FUREUkkhTiOToOIiCRR1aeXEeCWjrsMERGRYav6EE/hoMvpIiKSQApxAtyq/jSIiEgCVX16ZSkqxEVEJJGqPr0yFAhMHzETEZHkqfoQNwC1xEVEJIGUXurYJiIiCVX1IW44msZMRESSqKpD3N2jy+kKcRERSZ4qD3G1xEVEJLmqO8SJQlwtcRERSaCqDvGg53K6WuIiIpJAVR3i7pAytcRFRCSZqjrEgyAIF/Q5cRERSaAqTy8Pv6glLiIiCVTVIV4ohi1x0z1xERFJoKoO8cCjEE8pxEVEJHmqOsS9GF5Ot+o+DSIiklBVnV69LXE1xEVEJIGqO8R7eqenqvo0iIhIQlV1ehVdHdtERCS5qjrEPdBHzEREJLmqOsS33xNXiIuISPJUdYi7R73TNWKbiIgkUFWnV1DsGXY13jpERER2RVWHOIVuAGrz22IuREREZPiqOsR7PmLWNWZazJWIiIgMX1WHeKFYAHRPXEREkqmq08uLRQA6CzEXIiIisguqO8QJL6c31NbEXImIiMjwVXWIB1FLXMOuiohIElV1enkQhrgpxEVEJIGqOr16eqebpWOuREREZPiqPMR1OV1ERJKrqtPLCp3hV7XERUQkgao6xHtGXc0W2+MtREREZBdUdYj3TIBSrBsfcyUiIiLDV90h3tuxrapPg4iIJFRVp1d7dzhUWyqlacxERCR5qjrEs5kwvAvFmAsRERHZBVUd4hbdE6/Jqne6iIgkTybuAmIVhTjocrqIJMO2bdtYv349+Xw+7lJkBDU0NDBz5kxSwxy3pKpDvGcCFDOFuIiMftu2bWPdunXstdde1NfX63fXHiIIAlavXs3GjRuZMmXKsF5b1ZfT6W2I6z+CiIx+69evZ6+99mLMmDEK8D1IKpVi6tSpbN26dfivLUM9idHzOXH9ZxCRJMjn89TX18ddhpRBNpulUCgM+3VVHeLb74lX92kQkeRQo2PPtKs/16pOL3fdExcRkZHV2NjIiy++WJFjVXWI994UV4iLiOy2uXPnUl9fT2NjI9OmTWPRokW0tbXtsM/ixYs57rjjaGpqYuzYsbzzne9k6dKlO+yzbds2Lr30UmbPnk1jYyP77rsvl156KRs3bixr/Q888AAzZ87c7fdpa2tjn332GYGKhlbVId5zT1yfMBMRGRl33HEHbW1tPPHEEzz++ON84Qtf6H1uyZIlnHjiiZx55pmsWbOGl156iYMPPpijjjqqt+Way+U4/vjjefbZZ7nnnnvYtm0bS5YsYeLEiTzyyCNxfVu9duW+dTlVdYhbb4ZX9WkQERlx06ZN46STTuKJJ57o3faJT3yCCy+8kEsuuYSmpiYmTJjA5z73OY444giuuuoqAH70ox+xYsUKfvnLX3LQQQeRSqWYMmUKn/70pzn11FMHPNbixYs57LDDGDt2LIcddhiLFy/ufe6YY47h05/+NEcddRRNTU2ceOKJA7bo29vbOeWUU1izZg2NjY00NjayZs0arrrqKs455xzOP/98mpubue6663jkkUc48sgjGTduHNOnT+fiiy8ml8v1vpeZsXz5cgAWLVrERRddxGmnnUZTUxOHH344f/3rX0fgDIeqPL2iuUh1OV1EZEStWrWKu+++m/322w+Ajo4OFi9ezLvf/e7X7Hvuuedy3333AXD//fdz8skn09jYWNJxNm3axGmnncZHP/pRWlpauOyyyzjttNNoaWnp3eeGG27ghz/8IevXryeXy/GVr3zlNe/T0NDA3XffzYwZM2hra6OtrY0ZM2YA8Ktf/YpzzjmHLVu28L73vY90Os1//dd/sXHjRpYsWcJvfvMbvvWtb+20xptuuokrr7ySzZs3s99++3H55ZeX9L2VoqoHe+ntna4QF5EE+uwdz7J0zbayHuOgGc1c+c7Xlbz/WWedhZnR1tbGcccdx2c/+1kgDNsgCJg+ffprXjN9+vTe1nFLSwtvfvObSz7er3/9a+bNm8cFF1wAwHnnncc111zDHXfcwaJFiwB4//vfz/z584HwD4bbb7+95PcHOPLIIznrrLMAqK+v36G+uXPn8o//+I88+OCDXHrppQO+/l3vehcLFy4E4H3vex+XXXbZsI4/mKpuiTv6nLiIyEi67bbbaG1t5YEHHmDZsmW94Tx+/HhSqRRr1659zWvWrl3LpEmTAJg4ceKA++zMmjVrmDNnzg7b5syZw+rVq3vXp02b1rs8ZsyY13S2G8qsWbN2WH/hhRc4/fTTmTZtGs3NzXzqU58atNPd7h5/MGqJA+rZJiJJNJwWcqUdffTRLFq0iI997GPcdtttNDQ0cOSRR3LLLbdw7LHH7rDvzTffzPHHHw/ACSecwBVXXEF7ezsNDQ1DHmfGjBm88sorO2xbsWIFJ5988rBr3lmDrv/2j3zkIxx66KHceOONNDU18bWvfY1bb7112McbCdXdEu8ZsU3ziYuIjLhLL72U++67jyeffBKAL37xi1x//fVcc801tLa2snnzZq644gqWLFnClVdeCcAFF1zArFmzOPvss1m2bBlBENDS0sLnP/957rrrrtcc49RTT+WFF17ghhtuoFAo8LOf/YylS5dy+umnD7veqVOn0tLSMuTwp62trTQ3N9PY2MiyZcv49re/PexjjZSqDnG1xEVEymfy5MlceOGFXH311QC89a1v5d577+UXv/gF06dPZ86cOTz++OM89NBDzJs3D4Da2lruv/9+DjjgAN7xjnfQ3NzMwoUL2bhxI4cffvhrjjFx4kTuvPNOvvrVrzJx4kS+9KUvceedd/Zenh+OAw44gPPOO4999tmHcePGsWbNmgH3+8pXvsINN9xAU1MTH/zgB3nPe94z7GONFOv9rHRCLFiwwB999NERea/F993KW/7wAVad9XNmHnLCiLyniEi5PPfccxx44IFxlyFlsrOfr5k95u4LBnqNWuIAVt2nQUREkqmq0yubDz+aod7pIiKSRFUd4sVULQAWjK5h9EREREpR1SFONIuZ1zbFXIiIiMjwKcRB98RFRCSRqju9vBgtVPdpEBGRZKru9OoZ7CWdjrkQERGR4avyEI8up1f5aRARkWQqa3qZ2clm9ryZLTezTw7wfK2Z/Sx6/o9mNrec9bzm+EF4Od1SCnEREUmesqWXmaWBbwKnAAcB55nZQf12+wCw2d33A/4L+I9y1TOwqHe66XK6iMjumjt3LvX19TQ2NjJt2jQWLVr0mhm7Fi9ezHHHHUdTUxNjx47lne98J0uXLt1hn23btnHppZcye/ZsGhsb2Xfffbn00ksHnSlsJDzwwAPMnDlzRN7rmGOO4fvf//6IvNdgytkEXQgsd/cX3T0H3ASc2W+fM4Hro+VbgeOtkiOv9E6Aopa4iMhIuOOOO2hra+OJJ57g8ccf5wtf+ELvc0uWLOHEE0/kzDPPZM2aNbz00kscfPDBHHXUUbz44osA5HI5jj/+eJ599lnuuecetm3bxpIlS5g4cSKPPPJIXN/WqFXO9NoLWNlnfVW0bcB93L0AbAUmlrGmHZjrcrqISDlMmzaNk046iSeeeKJ32yc+8QkuvPBCLrnkEpqampgwYQKf+9znOOKII7jqqqsA+NGPfsSKFSv45S9/yUEHHUQqlWLKlCl8+tOf5tRTTx3wWIsXL+awww5j7NixHHbYYSxevLj3uWOOOYZPf/rTHHXUUTQ1NXHiiScO2KJvb2/nlFNOYc2aNTQ2NtLY2MiaNWsIgoAvfvGL7LvvvkycOJFzzz2XTZs2AdDV1cX555/PxIkTGTduHIcddhjr1q3j8ssv5/e//z0XX3wxjY2NXHzxxSN3YvtJRHqZ2YfM7FEze3TDhg0j9r41TZN5PrM/mZr6EXtPERGBVatWcffdd7PffvsB0NHRweLFi3n3u9/9mn3PPfdc7rvvPgDuv/9+Tj75ZBobG0s6zqZNmzjttNP46Ec/SktLC5dddhmnnXYaLS0tvfvccMMN/PCHP2T9+vXkcjm+8pWvvOZ9GhoauPvuu5kxYwZtbW20tbUxY8YMvv71r3Pbbbfx4IMPsmbNGsaPH89FF10EwPXXX8/WrVtZuXIlLS0tXHvttdTX1/Pv//7vvO1tb+Mb3/gGbW1tfOMb3xj2+StVpmzvDKuBWX3WZ0bbBtpnlZllgLFAS799cPfvAt+FcBazkSrwkHf8Lbzjb0fq7UREKuvuT8KrT5f3GNPeAKd8seTdzzrrLMyMtrY2jjvuOD772c8CYdgGQcD06dNf85rp06f3to5bWlp485vfXPLxfv3rXzNv3jwuuOACAM477zyuueYa7rjjDhYtWgTA+9//fubPnw+EfzDcfvvtJb//tddeyze+8Y3ee+VXXXUVs2fP5sc//jHZbJaWlhaWL1/OG9/4xmHVPVLK2RL/EzDPzPY2sxrgvUD/M3c78HfR8jnA/3rS5kYVEZFet912G62trTzwwAMsW7asN5zHjx9PKpVi7dq1r3nN2rVre+f/njhx4oD77MyaNWuYM2fODtvmzJnD6tXb24zTpk3rXR4zZsxrOtsN5pVXXuFd73oX48aNY9y4cRx44IGk02nWrVvHBRdcwEknncR73/teZsyYwSc+8Qny+XzJ7z0SytYSd/eCmV0M3AukgR+4+7NmdjXwqLvfDvwP8GMzWw5sIgx6EREpxTBayJV29NFHs2jRIj72sY9x22230dDQwJFHHsktt9zCscceu8O+N998M8cffzwAJ5xwAldccQXt7e00NDQMeZwZM2bwyiuv7LBtxYoVnHzyycOueaB+1bNmzeIHP/gBRx111ICvufLKK7nyyit5+eWXOfXUU9l///35wAc+ULHZMct6T9zd73L3+e6+r7v/e7TtM1GA4+5d7v5ud9/P3Re6+4vlrEdERCrn0ksv5b777uPJJ58E4Itf/CLXX38911xzDa2trWzevJkrrriCJUuWcOWVVwJwwQUXMGvWLM4++2yWLVtGEAS0tLTw+c9/nrvuuus1xzj11FN54YUXuOGGGygUCvzsZz9j6dKlnH766cOud+rUqbS0tLB169bebR/+8Ie5/PLLe/9Q2LBhA7/61a8A+O1vf8vTTz9NsVikubmZbDZLKuooPXXq1N4e9+WUiI5tIiKSPJMnT+bCCy/k6quvBuCtb30r9957L7/4xS+YPn06c+bM4fHHH+ehhx5i3rx5ANTW1nL//fdzwAEH8I53vIPm5mYWLlzIxo0bOfzww19zjIkTJ3LnnXfy1a9+lYkTJ/KlL32JO++8s/fy/HAccMABnHfeeeyzzz6MGzeONWvWcMkll3DGGWdw4okn0tTUxBFHHMEf//hHAF599VXOOeccmpubOfDAAzn66KN7781fcskl3HrrrYwfP56PfvSju3oKh2RJuwW9YMECf/TRR+MuQ0Sk4p577jkOPPDAuMuQMtnZz9fMHnP3BQO9Ri1xERGRhFKIi4iIJJRCXEREJKEU4iIiIgmlEBcRSZCkdUaW0uzqz1UhLiKSENlsls7OzrjLkDLI5/NkMsMff00hLiKSEFOmTGH16tV0dHSoRb4HCYKAdevWMXbs2GG/tpwToIiIyAhqbm4GwvHCKz1Gt5RXQ0PDLg1QoxAXEUmQ5ubm3jAX0eV0ERGRhFKIi4iIJJRCXEREJKEU4iIiIgmVuFnMzGwD8MqQO5ZuErBxBN+vWuk87j6dw92nc7j7dA5330ifwznuPnmgJxIX4iPNzB7d2RRvUjqdx92nc7j7dA53n87h7qvkOdTldBERkYRSiIuIiCSUQhy+G3cBewidx92nc7j7dA53n87h7qvYOaz6e+IiIiJJpZa4iIhIQlVNiJvZyWb2vJktN7NPDvB8rZn9LHr+j2Y2N4YyR7USzuFlZrbUzJ4ys9+Y2Zw46hzNhjqHffY728zczNRLeAClnEczOzf69/ismd1Q6RpHuxL+P882s9+a2ePR/+lT46hztDKzH5jZejN7ZifPm5ldE53fp8zsTWUpxN33+AeQBv4K7APUAE8CB/Xb55+Aa6Pl9wI/i7vu0fQo8RweC4yJlj+iczj8cxjt1wT8DngYWBB33aPtUeK/xXnA48D4aH1K3HWPpkeJ5/C7wEei5YOAl+OuezQ9gLcDbwKe2cnzpwJ3AwYcAfyxHHVUS0t8IbDc3V909xxwE3Bmv33OBK6Plm8Fjjczq2CNo92Q59Ddf+vuHdHqw8DMCtc42pXy7xDg34D/ALoqWVyClHIePwh80903A7j7+grXONqVcg4d6JkubSywpoL1jXru/jtg0yC7nAn8yEMPA+PMbPpI11EtIb4XsLLP+qpo24D7uHsB2ApMrEh1yVDKOezrA4R/hcp2Q57D6JLbLHf/dSULS5hS/i3OB+ab2R/M7GEzO7li1SVDKefwKuB8M1sF3AX8c2VK22MM93fmLtF84jLizOx8YAFwdNy1JImZpYD/BBbFXMqeIEN4Sf0YwitCvzOzN7j7ljiLSpjzgOvc/atmdiTwYzN7vbsHcRcm21VLS3w1MKvP+sxo24D7mFmG8PJRS0WqS4ZSziFmdgJwOXCGu3dXqLakGOocNgGvBx4ws5cJ76Pdrs5tr1HKv8VVwO3unnf3l4AXCENdQqWcww8ANwO4+xKgjnBMcClNSb8zd1e1hPifgHlmtreZ1RB2XLu93z63A38XLZ8D/K9HvRMEKOEcmtmhwHcIA1z3IF9r0HPo7lvdfZK7z3X3uYT9Cs5w90fjKXfUKuX/822ErXDMbBLh5fUXK1jjaFfKOVwBHA9gZgcShviGilaZbLcDF0a91I8Atrr72pE+SFVcTnf3gpldDNxL2CvzB+7+rJldDTzq7rcD/0N4uWg5YWeF98ZX8ehT4jn8MtAI3BL1CVzh7mfEVvQoU+I5lCGUeB7vBU40s6VAEfi4u+vKWqTEc/gvwPfM7P8QdnJbpIbNdmZ2I+EfipOifgNXAlkAd7+WsB/BqcByoAN4f1nq0M9EREQkmarlcrqIiMgeRyEuIiKSUApxERGRhFKIi4iIJJRCXEREJKEU4iIiIgmlEJfEM7OimT3R5zF3kH3bRuB415nZS9Gx/hwNSTnc9/i+mR0ULX+q33OLd7fG6H16zsszZnaHmY0bYv9DdmW6STObbmZ3RsvHmNnW6LjPmdmVu/B+Z/RMjWlmZ/Wcp2j96mhUwN0S/QzPGWKfB4YzWl70vd9Zwn4DTmFpZl8xs+NKPZ4IKMRlz9Dp7of0ebxcgWN+3N0PAT5JOErdsLj7P7j70mj1U/2ee8vulwdsPy+vJxzA6KIh9j+EcHCK4boM+F6f9d9H52YB4QQaw5pH2d1vd/cvRqtnEU6D2fPcZ9z9/l2ocTS5DhhoQpavE/57EimZQlz2OGbWaGa/iVrJT5vZa6b7jFqPv+vTUn1btP1EM1sSvfYWM2sc4nC/A/aLXntZ9F7PmNml0bYGM/u1mT0ZbX9PtP0BM1tgZl8E6qM6fho91xZ9vcnMTutT83Vmdo6Zpc3sy2b2JzN7ysz+sYTTsoRoBiUzWxh9j4+b2WIz2z8aevNq4D1RLe+Jav+BmT0S7TvQtKkAZwP39N/o7u3AY8B+USv/4ajeX5rZ+KiWj5rZ0mj7TdG2RWb2DTN7C3AG8OWopn37nIOTzeyWPuemtxU83J+hmX0mOpfPmNl3zXaYgviCPv9GFkb7l3peBrSzKSzd/RVgoplNG877SZUr98TpeuhR7gfhsJpPRI9fEg4n3Bw9N4lw2MOe0Qnboq//AlweLacJJx+ZRBjKDdH2fwU+M8DxrgPOiZbfDfwReDPwNNBAOPTss8ChhAH3vT6vHRt9fQBY0LemPvv01Pgu4PpouYZwWsN64EPAFdH2WuBRYO8B6mzr8/3dApwcrTcDmWj5BODn0fIi4Bt9Xv954PxoeRzhJCIN/Y6xN/BYn/VjgDuj5YnAy8DrgKeAo6PtVwNfi5bXALU9x+hfR99z3Xc9+hmv6POz+jZw/i7+DCf02f5j4J19fkbfi5bfDjwz2Hnp970vAL4/yL/ZuT3v12/794Cz4/4/pUdyHlUxdrrs8To9vHwLgJllgc+b2duBgLAFOhV4tc9r/gT8INr3Nnd/wsyOJrx0+4eoMVZD2IIdyJfN7ArCCSE+QDhRxC89bH1iZr8A3kbYQv2qmf0H4S/43w/j+7ob+G8zqyW8/Po7d+80sxOBN/a5pzuWcIaul/q9vt7Mnoi+/+eA+/rsf72ZzSMcEzu7k+OfCJxhZh+L1uuA2dF79ZjOayfFeJuZPU547r9IOKPYOHd/MHr+esI/KiAM95+a2W2Ek5aUxMOxv+8B3mlmtwKnAZ8gnP621J9hj2PN7BPAGGAC4R9gd0TP3Rgd73dm1mxhv4KdnZe+9T0K/EOp308f64EZu/A6qVIKcdkTvQ+YDLzZ3fMWTutZ13eH6Jfy2wl/+V9nZv8JbAbuc/fzSjjGx9391p4VMzt+oJ3c/YXonvCpwOfM7DfufnUp34S7d5nZA8BJwHuAm3oOB/yzu987xFt0uvshZjaGcKKLi4BrgH8Dfuvu77KwE+ADO3m9EbYKnx/sGPQ7t4T3xE/vfROzsYO8/jTCVu47gcvN7A2D7NvfTcDFhJemH3X31uhSeKk/Q8ysDvgW4VWRlWZ2FTt+P/0nl3B2cl7MbOowat+ZOsJzKlIS3ROXPdFYYH0U4McCc/rvYGZzgHXu/j3g+8CbCKf+PMrMeu5xN5jZ/BKP+XvgLDMbY2YNhJfCf29mM4AOd/8J4SxvA3XyykdXBAbyM8LZj3pa9RAG8kd6XmNm86NjDsjdO4CPAv9iZhnC89Mzr/GiPru2Et5W6HEv8M8994gtnGq2vxcILw3vlLtvBTZb1O8AuAB40MxSwCx3/y3hZe+xhLci+upfU18PEp7PD7L9D5zh/gx7AntjdO+8f4/1nj4MbyWcSnIrpZ2XXTUfeGbIvUQiCnHZE/0UWGBmTwMXAssG2OcY4Mnosu97gP929w2EoXajmT1FeBn2gFIO6O5/JrzP+gjhPfLvu/vjwBuAR6LL2lcCnxvg5d8FnrKoY1s//4/wEvH97p6Ltn0fWAr82cKPKX2HIa6qRbU8BZwHfAn4QvS9933db4GDejq2EbbYs1Ftz0br/d+3HfhrT2gO4u8Ib0E8RdgL/mrCe/U/iX5OjwPXuPuWfq+7Cfh41IFs337HLgJ3AqdEXxnuzzA63vcIg/NewtssfXVF5+lawtsmUMJ5sbDT4vcHOqaFU1guAfY3s1Vm9oFoe5awk6Tmj5eSaSpSEdktZvYuwlsXV8RdS5JF5/FN7v7puGuR5NA9cRHZLe7+SzObGHcde4AM8NW4i5BkUUtcREQkoXRPXEREJKEU4iIiIgmlEBcREUkohbiIiEhCKcRFREQS6v8D9KCsfXzcVVkAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "estimate_model(sgd_clf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3 基于 `在线学习` 的 `SGD` 分类模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.3.1 在线学习算法简介及理论\n",
    "\n",
    "&emsp;&emsp;在线学习，是用数据实例持续地进行训练，可以一次一个或一次几个实例（称为`小批量`）。每个学习步骤都很快且`廉价`，所以系统可以动态地学习收到的最新数据。  \n",
    "&emsp;&emsp;在线学习很适合系统接收连续流的数据（比如，股票价格），且需要自动对改变作出调整。如果计算资源有限，在线学习是一个不错的方案：一旦在线学习系统学习了新的数据实例，它就不再需要这些数据了，所以扔掉这些数据（除非你想滚回到之前的一个状态，再次使用数据）。这样可以节省大量的空间。   \n",
    "&emsp;&emsp;在线学习算法也适用于在超大数据集（一台计算机不足以用于存储它）上训练系统（这称作核外学习，out-of-core learning）。算法每次只加载部分数据，用这些数据进行训练，然后重复这个过程，直到使用完所有数据。  \n",
    "&emsp;&emsp;在线学习系统的一个重要参数是，它们可以多快地适应数据的改变：这被称为学习速率。如果设定一个高学习速率，系统就可以快速适应新数据，但是也会快速忘记老数据。相反的，如果设定的学习速率低，系统的惰性就会强：即，它学的更慢，但对新数据中的噪声或没有代表性的数据点结果不那么敏感。  \n",
    "&emsp;&emsp;在线学习的挑战之一是，如果坏数据被用来进行训练，`系统的性能就会逐渐下滑`。如果这是一个部署的系统，用户就会注意到。例如，坏数据可能来自失灵的传感器或机器人，或某人向搜索引擎传入垃圾信息以提高搜索排名。要减小这种风险，需要密集监测，如果检测到性能下降，要快速关闭（或是滚回到一个之前的状态）。你可能还要监测输入数据，对反常数据做出反应（比如，使用异常检测算法）。   \n",
    "\n",
    "\n",
    "&emsp;&emsp;许多在线学习问题本质上可以（重新）表述为在线`凸优化`（OCO）任务。下面，我们介绍OCO的一些基础知识。\n",
    "\n",
    "&emsp;&emsp;在线凸优化任务通常有两个主要元素组成：凸集S和凸代价函数 $l_t(\\cdot)$。在每个时间步长 $t$，在线算法决定选择一个权重向量 $w_t\\in S$ ；此后，他遭受损 失$l_t(w_t)$ ，该损失是基于在 $s$ 上定义的凸成本函数 $l_t(\\cdot)$ 计算的。在线算法的目标是选择一系列决策 $w_1,w_2,...$ 使得事后的损失最小化。$$R_T=\\sum_{t=1}^{T}l_t(w_t)-\\mathop{inf}\\limits_{w^* \\in S}\\sum_{t=1}^{T}l_t(w^*) $$\n",
    "其中 $w^*$ 是 $S$ 上最小化凸目标函数 $\\sum_{t=1}^T l_t(w)$ 的解。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.3.2 基于 `在线学习` 的 `SGD` 分类模型训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import HashingVectorizer\n",
    "from sklearn.linear_model import SGDClassifier\n",
    "\n",
    "# 调用 stream_docs 流式读取数据，若不存在已预处理的数据，函数自动对数据进行预处理\n",
    "doc_stream = stream_docs(file_processed)\n",
    "\n",
    "vect = HashingVectorizer(decode_error='ignore', n_features=2**21, tokenizer=tokenizer)\n",
    "\n",
    "sgd_online  = SGDClassifier(loss='log', random_state=1)\n",
    "\n",
    "# 一共12万条数据，每次小批量处理1000条，一共分为120批次\n",
    "# 按照7:3划分训练集和测试集，训练集85批次，测试机35批次\n",
    "\n",
    "classes = np.array([0, 1])\n",
    "for _ in range(85):\n",
    "    X_train, y_train = get_minibatch(doc_stream, size=1000)\n",
    "    if not X_train:  break\n",
    "    X_train = vect.transform(X_train)\n",
    "    sgd_online.partial_fit(X_train, y_train, classes=classes)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.3.3 SGD 分类模型在线学习的各项指标及ROC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将剩下的35000条作为测试集\n",
    "X_test, y_test = get_minibatch(doc_stream, size=35000)\n",
    "X_test = vect.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练集上的准确率：0.92600,\t测试集上的准确率：0.91988\n",
      "训练集上的精度：  0.89583,\t测试集上的精度：0.89059\n",
      "训练集上的召回率：0.96138,\t测试集上的召回率：0.95817\n",
      "训练集上的F1得分：0.92745,\t测试集上的F1得分：0.92314\n",
      "训练集上ROC面积: 0.97636, 测试集上ROC面积: 0.97252\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAHgCAYAAAC1uFRDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABE7UlEQVR4nO3deZxcVZ3//9enqjvpTi9ZOvvOEoQom4QEBGUJskPgBwJBYOI44gIDGUYdvwKCjAuj4jgIiuCgyBhZVCAgy4AC6iQGg4QlIWDYQhZI0lk7W3dVfX5/3NudSqe763bS1bdv1/v5eNSjbt26VffdtwOfPveee465OyIiIpI8qbgDiIiIyO5RERcREUkoFXEREZGEUhEXERFJKBVxERGRhFIRFxERSaiyuAN01uDBg338+PFxxxAREekWzz///Bp3H9LWe4kr4uPHj2f+/PlxxxAREekWZvZOe+/pdLqIiEhCqYiLiIgklIq4iIhIQqmIi4iIJJSKuIiISEKpiIuIiCSUiriIiEhCqYiLiIgklIq4iIhIQqmIi4iIJJSKuIiISEKpiIuIiCSUiriIiEhCqYiLiIgkVNGKuJndaWarzOyVdt43M7vZzJaY2Utm9uFiZREREemNitkS/zlwcgfvnwJMCB+XAj8uYhYREZFep2hF3N3/CKztYJNpwC888BdggJmNKFYeERGRLuUOuRzkspDNQNM22L4Jtq7rtghl3banXY0C3s17vSxctzKeOCKSdLPmLeWhBcu77gvdMRxwDDByWLgMTsqb32veDlLkAFrWWfgdZWTCz+R2vIeH3wnmOVLhvlIt652K3DbcjLRnKCND2rOUeyN9fXvevndkA0h5qwy0+jnCDINy9TRYDUYu3Ge4X3dSLT9r8N7QzHtsSA8IvsfZ9Tubf47mdb7z+tbHaXhmBWvTdTu2cd8l745jmNvpuwZl17DNKslautXPtfPvo3lfOx/nHce4+bnMm9re3mHX4xc8UuG2bdmarqHy2mW7+Y+uc+Is4pGZ2aUEp9wZO3ZszGlESpA7ZJugaXPQ6shlwXPhI285l4NcEzRtDVspTcHnttRDug/geZ8Llv/v76uZ/059UETy/medIsfAbD1bUtWkPEuKHGmypD3LkOz7NKSqKfMM/XPrqfQtNFkfhmzexmfJUleZYkj2fbam+uEYKc+FRSl4bn5d7k2kyJIm27IuRY60Z+lDY9xHPRa5lvId/ilhwbKRo8K3sz41cKfS3VLWbOcSmP8gf3uzlt/J3k1/Z016aNul24IymSONW9lO+2lIVVOXXcN76ZEt39e8jx3lOe9PCmv+EyXIl2uVPWdpMqR3yrejbKd2/n5r/bPt2Db4njIGDRzAad30+4qziC8HxuS9Hh2u24W73w7cDjBp0qT2//wRKZKuaOFZWCTKPEMZTaQ8KB5pcqQ8S6VvwXDSniVNhr6+nQrfimOkPUsZGQZl69lmFZSRocwz1GVXsS1V2dKKSZENi5SHRS/D0Mx7NKRqGJ5dwTarBGgpZPlFzXBGZJexPjUoLJYZqnMNRS9mR4WP3bHZqshaGf1ym1mTHsLWvv2oraqgrqYfpMbCppVQty9YGlJpsFTwSKXDdWVQ1jfvdbjOUoBD42aoHgZY+FnC5fB183K761LhMsEp1prhwX6at83fvnnbndaH6zLboV8dpMshVR4896na+TMFMzS/bzu/X9Z3x89sRoqOr7MO2M3fVTG+T026eIv4bOByM7sHmAJscHedSpfiyDRCZhs0NsD2huB/qJtWwubVwf+8so2w9i2o6M/CZWtYvXIpZZ6lycpJk6G6YSuXkOVDfVezPjUwbLEFrcIUOWpyG+njjWSsjGHZ99hmfYHgtGZzoS7qj0cab25HWmqnk6NpsuRIUZ8ezKjsGlaUjSZHioyV7bR9jhRr0kOo9gbeTw8nSxlZK6PCt7Ix1Z8Gq6GfN7AmPazlc77jz4CW1lOOFBW+lXWpQWStLPyeNOa5sGW840Rmc4vouAOGc8Yho4MfprmINRef8n5h4S0LH80FN0VV3jEYXtQjLNIzFa2Im9mvgGOBwWa2DLgOKAdw99uAR4FTgSXAFuBTxcoiCeAeFNkta4Pn7ZuCZ89BLhM8shloeC/YduPyoDXStDVoLWUbg8eaJVDRH95fCFvWQFkFbFvfqSj7ejl7kaKfbWddaiCN1pdsnzT9KisYUFnBuK3LYeBeYautuaCMCfZTMwIqB1CxZS0MHJ/Xskvv3NorqwhbVXnvN22BqsHBe6lySJcFp637DdrR+kqVBe+XVwbL6T5Q1pey5tZeB6rD5wERjsEBnTpiIhKXohVxd59e4H0HLivW/qW4Ojq9nPYmqnOb6OebGZBdx4jsckZkllPh2xiYXUNf387Q7Hv08e3kSFFGhgG59buVI4ex3SrYbn3JUEbO0lTktrK8bAwpq2Wj1/J8ZjDVbCU3cG+arJwUOValR7AhNYBtVsnmVBVN1ocmK6eRvmDGtENGceGUsQzcg2MkIlJsiejYJvGbNW8pf3h+IQNy6xiTeYfM6iUcbRkm1mxjVGYpW1P9qMltZERmOVW+ud3vyZFibaqO7VbBunQ1q9PD2JSqJUMZKXJsTA3gvfQImqwPZWTYkBoQnIy24KS0YzSkathi/diSqiJL2Y5rju0ZBNMOGcVFU3QFTUR6FxVx2VXDKvj7/8L7i2DtGzS8+xLHbNnOhVa/Y5vy8NnqoLICrAEG7QV1R0PNyODUdt2+wenh2pHBqeXySlLA4Lxd7dN9P5WISK+jIl7q1r8Lf/9f3p/zS9Ib3qEPTdTmNuy0Scr7ssUH89KEL3DQwYdB1RAYOA5qRwfXbUVEJBb6P3CpadwCf38CVr0Kz93eMrLQMKDBK3ir7/682udA3i7fm4V9DmZbqh9AyzViERHpOVTES8GGZbz08K00vvs8k7bPa1m9NjWI/80cy0O5o0iPncyph+7NhVPGcmCMUUVEJDoV8V7swWfmUT3vPzlh62McBDR6mj/0O5E3y/fjpb4f5v30iJ16YouISLKoiPdGW9bCU9dx1t9+AcDzfSdzf83FHHT4sVw4ZSzHxxxPRES6hop4LzFr3lIeeeFtLt3wX3xs6x9IkWMZQ/jZoKu49oovcFjcAUVEpMupiCdc86ArK95+lSf7fJkKa2JVeig/7X8FL/adxLRDRsUdUUREikRFPAE6Gh3tubfWMCP9BPf2vTtYMfU6hn70Kr7ajflERCQeKuI9UOuiPe+ttQBM2WvQTtsNzK7hz1VfYVR2WTCYyuk/gH2O68akIiISJxXxHuihBctZtHIjE0fUAkHx3qUH+dq34I4LIbsWDpsRFPAIk2CIiEjvoSLeQ00cUcu9nz2y7TffXwh3TIXMVviHh2Gvj3VvOBER6RE6mvtdehp3ePFe+PFHgmk6P/kbFXARkRKmlngPkX8dPP9UeovN9XDr5GCObIDP/B5G6cYxEZFSpiIeo/zCnd95beKI2p1vDcvl4I5jgwK+/+lw9m3QtyaGxCIi0pOoiMcovwNbm53XALJNcPfZsH4pTL0OPnpVPGFFRKTHURGPWYcd2Nzh3ovh7T/BR78IR/9L94YTEZEeTR3bYjJr3tKWU+htcofH/g1efwzGHQVTr9UtZCIishO1xGMwa95SvvrAywBtD4uay8Evz4U3fg/jjoZ/mN3NCUVEJAnUEu9m+QX8W2cf2PYUoPd+Mijg+50SFPBUuptTiohIEqgl3s2ae6O3WcCbT6G/9iiUVcD0X+kUuoiItEtFvJs03062aOVGpuw1qO0W+OP/D577CXzgNDj3ThVwERHpkIp4kTUX7/z7wNu8Dr5sPsz7MYz9CJz/P5DSlQ4REemYiniR5be+27wPHOCdOfCr6VA5EC74pQq4iIhEoiLexVpPI9o8mEu794K/8YdgMJc+NXDRb6DfoLa3ExERaUVFvAu0N3wqsOsQqvk2roD/OSdY/vT/wrCJRc8qIiK9h4r4HmjreneHp83z1b8BPzoSPAfTblUBFxGRTlMR3wORrne3ZfsmuO8fINsIF94H+51U3KAiItIrqYjvoQ6vd7cll4VfngfvvwzTfqQCLiIiu03doLvb3Ftg6Rw4bAYc+sm404iISIKpiHendW/Dk1+DQfvAqTfFnUZERBJORby7bF4T3EoGQUe2tK5kiIjInlEl6Q65HNxxHGxYBhfMgnGduIYuIiLSDrXEu8NL98D6pTD5s7D/aXGnERGRXkJFvNgy2+GP34WaEXDSt+JOIyIivYhOpxfb3Ftg7Ztw9u0aE11ERLqUqkoxucPf7oaRh8LB58edRkREehkV8WJ64W5Y9xYc+Im4k4iISC+kIl4s2SZ4+ttQVhEM7CIiItLFVMR306x5S1smPmnT3Ftg0wo47SboU9V9wUREpGSoiO+m5qlH25xmtGkbvPybYPngC7sxlYiIlBIV8T0wZa9Bbc9c9sy3gglOTvmOeqSLiEjRqMIUw6KHgufJl8abQ0REejUV8a62+vVgopMDzgSzuNOIiEgvpiLe1V6+L3g+/pp4c4iISK+nIt7V/v4kDBgHQz4QdxIREenlVMR3Q7u3l61ZAisXwAfP7vZMIiJSelTEd0O7t5f9/uvB8+Gf7uZEIiJSilTEd9Mut5e5w9K5UDUEBrRx25mIiEgXUxHvpHZPpb/5DGxeDUd8odsziYhIaVIR76R2T6UvfCB4Pui8bk4kIiKlSkV8N7Q5Utsrv4XaUdB/dDyhRESk5KiId4UNy6BxE+xzfNxJRESkhKiId4W/3R08H3xBvDlERKSkqIh3hZfvg4oBMP7ouJOIiEgJURHfU5vrYe1bMO6ouJOIiEiJURHfU3+5FXD4yOVxJxERkRKjIr6nFj0EZZUw9si4k4iISIlREd8TuRzUL4HRkzTtqIiIdDsV8T2x6MHgWa1wERGJgYr4nmgu4kf/S6wxRESkNKmId8Iu46a/9wrUjIQ+/eILJSIiJUtFvBN2Gjd9w3JY+wbsd2LMqUREpFSpiHdSy7jprz8WrDh4eryBRESkZKmI764X74GqoTBmStxJRESkRKmI766VL0HNMN1aJiIisVERj2inTm1b1kJ2Owz7ULyhRESkpKmIR7RTp7bFvwtW7ndyjIlERKTUqYh3QkuntuYivvexseYREZHSVtQibmYnm9lrZrbEzL7SxvtjzexpM3vBzF4ys1OLmadL5LKwdA7UjIDKAXGnERGREla0Im5maeBW4BRgIjDdzCa22uwa4D53PxS4APhRsfJ0mfdfgW0b4IgvxJ1ERERKXDFb4pOBJe7+prs3AvcA01pt40BtuNwfWFHEPF1j8aPB875T480hIiIlr6yI3z0KeDfv9TKg9U3V1wP/a2b/DFQBJxQxT9dY83rwPHi/eHOIiEjJi7tj23Tg5+4+GjgVuNvMdslkZpea2Xwzm7969epuD7mThb+FfaZCujzeHCIiUvKKWcSXA2PyXo8O1+X7NHAfgLvPBSqAwa2/yN1vd/dJ7j5pyJAhRYpbWEVuS7CgAi4iIj1AMYv4X4EJZraXmfUh6Lg2u9U2S4GpAGZ2AEERj7mp3b4p2/4cLExsfWlfRESk+xWtiLt7BrgceAJ4laAX+kIzu8HMzgw3+1fgM2b2IvArYIa7e7Ey7akjtv0pWPjQufEGERERobgd23D3R4FHW637Wt7yIuCoYmboCs1Drg6sXgtVQ6CsT9yRREREYu/YlggPLVhOXxoZl3lLrXAREekxVMQjOmlMjhQ5GPKBuKOIiIgAKuKRjcm8EyxUD403iIiISEhFPKKRmXDcmvFHxxtEREQkpCJeQHOntv0aX4V0H6joH3ckERERQEW8oOZ5xD/IG5DuG3MaERGRHVTEI5iy1yD6ZTfC0APijiIiItJCRTyCitwWyGyDvT4WdxQREZEWKuIRjMwsCxb6j4o3iIiISB4V8QgmNr4cLIw8NN4gIiIieVTEIxiaXRksDNZALyIi0nOoiEcwrunNYKFPv3iDiIiI5FER70DzPeJpctB/TOEPiIiIdCMV8Q403yM+oWkxjPtIzGlERER2piJewPHjyoOFfoPjDSIiItKKingBExoXBwsjD4k1h4iISGsq4gUMyK0LFoZ9KN4gIiIiraiIF7B30+vBwqC94g0iIiLSiop4AWnPBgvllfEGERERaUVFvIAymnR7mYiI9Egq4gWMyKyAdHncMURERHahIl7AqMxS6FcXdwwREZFdqIh3xJ1qb9DpdBER6ZFUxDvQ17cHC8MPjDeIiIhIG1TEOzA8Gwy7ilm8QURERNqgIt6BEZmwiI/8cLxBRERE2lAWZSMzSwEHAyOBrcAr7r6qmMF6gnGZcArSEQfFG0RERKQNHRZxM9sH+DfgBODvwGqgAtjPzLYAPwHucvdcsYPGYWRmGavSwxhaOTDuKCIiIrso1BL/BvBj4LPu7vlvmNlQ4ELgYuCu4sSLV//sOlanhzE07iAiIiJt6LCIu/v0Dt5bBfygqwP1JCOyK3ijfL+4Y4iIiLRptzu2mdnHuzJIT5ShjKrcprhjiIiItGlPeqf/d5el6KH60Mi75ePjjiEiItKmQh3bZrf3FtCrxyKdNW8pF+Y24OgecRER6ZkKdWz7KHAR0NBqvQGTi5Koh5j9wlIuBMYOUc90ERHpmQoV8b8AW9z92dZvmNlrxYnUM/TzLQDs/4EDYk4iIiLStkK900/p4L2PdX2cnmNo9r1god+geIOIiIi0Q8OutqN/dl2wUDMi3iAiIiLtUBFvx75N4dWCwbpPXEREeiYV8XaFvdJrhscbQ0REpB0q4u0YkVlGhjSk0nFHERERaVPkIm5m13f0urep9C2UkY07hoiISLs60xJ/vsDrXmV0Zimr0pr6REREeq7IRdzdH+7odW+TIsfG1IC4Y4iIiLSr0LCrPwS8vffd/YouT9RDDMmuYnH5B9k37iAiIiLtKDRi2/xuSdHThFOnb01VxRxERESkfYVGbLsr/7WZ9XMPxyPtzZqCH3GL9Ys5iIiISPsiXRM3syPNbBGwOHx9sJn9qKjJ4rQ9mEN8U6o25iAiIiLti9qx7QfASUA9gLu/CPTesdM3rw6eUtUxBxEREWlfZ3qnv9tqVe+9ibpxMwBbdTpdRER6sEId25q9a2YfAdzMyoErgVeLFytmm1YCaomLiEjPFrUl/jngMmAUsAI4JHzdq61PaRpSERHpuSK1xN19DfDJImfpOdYsAWC79Y05iIiISPui9k7f28weNrPVZrbKzB4ys72LHS42ZUHx3pjqH3MQERGR9kU9nT4LuA8YAYwE7gd+VaxQsctsB6DJ+sQcREREpH1Ri3g/d7/b3TPh43+AimIGi9XaNwDIomlIRUSk5yo0dnpzz67HzOwrwD0EY6mfDzxa5GzxqQhPo5vFm0NERKQDhTq2PU9QtJur2Wfz3nPg/xUjVOwy21mvGcxERKSHKzR2+l7dFaRHWfsmmci30IuIiMQjcqUysw8BE8m7Fu7uvyhGqNhV1NI/tyHuFCIiIh2KVMTN7DrgWIIi/ihwCvBnoHcWcXdWlI2OO4WIiEiHovZOPxeYCrzn7p8CDgZ6703UuYx6pouISI8XtYhvdfcckDGzWmAVMKZ4sWKWy5A1FXEREenZol4Tn29mA4A7CHqsNwBzixUqdrkMObXERUSkh4s6dvoXwsXbzOxxoNbdXyperJite5us1cSdQkREpEOFBnv5cEfvufvfuj5SD9C3lgEb6+NOISIi0qFCLfGbOnjPgeO7MEvPkcuyrGxc3ClEREQ6VGiwl+O6K0iPsv4dmlKHxZ1CRESkQxqWrC25LDVsjDuFiIhIh6LeYlZinKXlpTnirIiIJIeKeGvueGY7b2/Ixp1ERESkQ5GKuAUuMrOvha/HmtnkCJ872cxeM7Ml4VSmbW1znpktMrOFZjarc/GLILMNw9niFUw7ZFTcaURERNoV9Zr4j4AcQW/0G4BNwG+Aw9v7gJmlgVuBjwPLgL+a2Wx3X5S3zQSC6UyPcvd1ZjZ0t36KrpTZBsCwugFcOGVszGFERETaF/V0+hR3vwzYBuDu64A+BT4zGVji7m+6eyNwDzCt1TafAW4Nvw93XxU5ebFsbwCgifKYg4iIiHQsahFvClvWDmBmQwha5h0ZBbyb93pZuC7ffsB+ZvZ/ZvYXMzs5Yp7iadwMgBX88UREROIV9XT6zcADwFAz+ybBrGbXdNH+JxBMczoa+KOZHeju6/M3MrNLgUsBxo4t8inubcE84qvTw4q7HxERkT0Udez0X5rZ8wTTkRpwlru/WuBjy9l5prPR4bp8y4B57t4EvGVmrxMU9b+22v/twO0AkyZN8iiZd9u29QA0pGqLuhsREZE9FbV3+s3AIHe/1d1viVDAISjEE8xsLzPrA1wAzG61zYMErXDMbDDB6fU3I2Yvjsx2ABqtb6wxRERECol6Tfx54Boze8PMvmdmkwp9wN0zwOXAE8CrwH3uvtDMbjCzM8PNngDqzWwR8DTwJXePd+aRpq0ANJo6tomISM8W9XT6XcBdZjYIOAf4DzMb6+4TCnzuUeDRVuu+lrfswFXho2doDHqnb7PKmIOIiIh0rLMjtu0L7A+MAxZ3fZweILxPfLtVxBxERESkY1GviX/HzP5OMNDLK8Akdz+jqMnisi2Y+GS7romLiEgPF/UWszeAI919TTHD9Ajb1rPF+pHVNXEREenhOiziZra/uy8m6Gk+1sx2uknb3f9WzHCxWLWIjAq4iIgkQKGW+FUEg6zc1MZ7TjCWeu9SOYgyb4o7hYiISEEdFnF3vzRcPMXdt+W/Z9ZLe35lm1iVHh53ChERkYKi9k6fE3Fd8mW3k7V03ClEREQKKnRNfDjBpCWVZnYowZCrALVAvyJni8ea13F650kGERHpXQpdEz8JmEEw7vn389ZvAr5apEzx6ldHumFj3ClEREQKKnRNvHmktnPc/TfdlClemUbNYCYiIolQ6HT6Re7+P8B4M9tlaFR3/34bH0u2rWvZbkPjTiEiIlJQodPpVeFzdbGD9Bhb17Oh78C4U4iIiBRU6HT6T8Lnr3dPnJjlcpDZyrYKTX4iIiI9X2fGTq81s3Iz+72ZrTazi4odrts1bgKgwrfEHERERKSwqPeJn+juG4HTgbcJZjP7UrFCxSYbjNS2Kj0i5iAiIiKFRS3izafdTwPud/cNRcoTr2wjAE0aO11ERBIg6ixmj5jZYmAr8HkzGwJsK/CZ5GncDEAWjdgmIiI9X6SWuLt/BfgIwTziTcBmYFoxg8UiPJ3exxtjDiIiIlJYpJa4mZUDFwEfMzOAZ4HbipgrHrkMAOvTusVMRER6vqin038MlAM/Cl9fHK77p2KEio1nAZ1OFxGRZIhaxA9394PzXv/BzF4sRqBY5YIinovc309ERCQ+UatV1sz2aX5hZnsD2eJEilF4On3R+7pPXEREer6oLfEvAU+b2ZsE05GOAz5VtFRxCTu2ZUgz7ZBRMYcRERHpWMEiHt5OtgGYDDTPDPKau28vZrBYbF0HwIHDK7lwytiYw4iIiHSsw9PpZvZPwELgh8ACYLy7v9QrCzhAOhjkZZPVxhxERESksEIt8ZnAB919dXgd/JfA7KKnikt4TTxr6p0uIiI9X6GObY3uvhrA3d8E+hY/UozCa+K6xUxERJKgUEt8tJnd3N5rd7+iOLFiEt5ippa4iIgkQaEi3nqmsueLFaRH2LgcgJxa4iIikgAdFnF3v6u7gvQIfaoA2Gr9Yg4iIiJSWKHe6XeY2Yfaea/KzP7RzD5ZnGgxyAQTs2kqUhERSYJCp9NvBb5mZgcCrwCrgQpgAlAL3EnQY713WPc2oCIuIiLJUOh0+gLgPDOrBiYBIwjmFH/V3V8rfrxuVtEfgAwq4iIi0vNFGnbV3RuAZ4obpQfIZWikDwTTrYqIiPRomq4rXy6r28tERCQxVMTz5TKahlRERBKjUxXLrJffe5XLarQ2ERFJjEhF3Mw+YmaLgMXh64PN7EdFTRaHXBM5nU4XEZGEiNoS/0/gJKAewN1fBD5WrFCxadpGRi1xERFJiMin09393Varsl2cJX6ZbWy3yrhTiIiIRBLpFjPgXTP7COBmVg5cCbxavFgxadpKo/WJO4WIiEgkUVvinwMuA0YBy4FDgC8UKVNsNi5fzJbGTNwxREREIonaEv+Au+80RrqZHQX8X9dHis+qpr7kaGTaIaPijiIiIlJQ1Jb4DyOuS7Tq3CY29xvNhVPGxh1FRESkoA5b4mZ2JPARYIiZXZX3Vi30vm7c1blNbErVxh1DREQkkkKn0/sA1eF2NXnrNwLnFitULNyp9ga2W9+4k4iIiERSaBazZ4Fnzezn7v5ON2WKR7YJAE19IiIiSRG1Y9sWM/su8EGC+cQBcPfji5IqDrmgiK9LDYo5iIiISDRRO7b9kmDI1b2ArwNvA38tUqZ4hC1xjZ0uIiJJEbWI17n7fwNN7v6su/8j0Hta4QC54P7wrEU9OSEiIhKvqBWrKXxeaWanASuA3nXeuWkLAIbHHERERCSaqEX8G2bWH/hXgvvDa4GZxQoVi/B0ek5d20REJCEiFXF3fyRc3AAcBy0jtvUeYRHflOofcxAREZFoCg32kgbOIxgz/XF3f8XMTge+ClQChxY/YjfJqWObiIgkS6GW+H8DY4DngJvNbAUwCfiKuz9Y5GzdK7M9eLLymIOIiIhEU6iITwIOcvecmVUA7wH7uHt98aN1s23rASjzpo63ExER6SEK3WLW6O45AHffBrzZKws48PTi1QCsT/euTvciItJ7FWqJ729mL4XLBuwTvjbA3f2goqbrRn99YyXHAR/5wMi4o4iIiERSqIgf0C0peoC67BoATjlodMxJREREoik0AUrvnvQkT8vsZX2q4w0iIiISUdRhV3u9FLlgIa3e6SIikgwq4qG0Z4OFlMZOFxGRZIhcxM2s0sw+UMwwcUqhIi4iIskSqYib2RnAAuDx8PUhZja7iLm6Xbr5dLrp5ISIiCRD1Ip1PTAZWA/g7gsI5hbvNQZnVwULaomLiEhCRC3iTe6+odW6XjVn51brFyyU94s3iIiISERRi/hCM7sQSJvZBDP7ITCniLm6Xbk3kqEM0mqJi4hIMkQt4v8MfBDYDswimJJ0ZpEyxWJkZplmMBMRkUSJ2uzc392vBq4uZpg4bUrV0pftcccQERGJLGpL/CYze9XM/t3MPlTURDFJk2VNakjcMURERCKLVMTd/TjgOGA18BMze9nMrin0OTM72cxeM7MlZvaVDrY7x8zczCZFTt7FUuTI6fYyERFJkMhVy93fc/ebgc8R3DP+tY62N7M0cCtwCjARmG5mE9vYrga4EpgXPXbXS3lO18RFRCRRog72coCZXW9mLwPNPdMLTfc1GVji7m+6eyNwDzCtje3+HfgPYFv02F0vRZacRqEVEZEEiVq17iQY6OUkdz/W3X/s7qsKfGYU8G7e62XhuhZm9mFgjLv/LmKOoglOp6slLiIiyRGpd7q7H9nVOzazFPB9YEaEbS8FLgUYO3ZsV0cBYERmGY4V5btFRESKocOWuJndFz6/bGYv5T1eNrOXCnz3cmBM3uvR4bpmNcCHgGfM7G3gCGB2W53b3P12d5/k7pOGDOn6HuSz5i1l6fZq+uUauvy7RUREiqVQS/zK8Pn03fjuvwITzGwvguJ9AXBh85vhMK6Dm1+b2TPAF919/m7sa488tGA5V1mG7IBeNRy8iIj0ch22xN19Zbj4BXd/J/8BfKHAZzPA5cATwKvAfe6+0MxuMLMzuyJ8VxpTvolhA2vjjiEiIhJZ1BHbPg78W6t1p7Sxbifu/ijwaKt1bd6a5u7HRsxSFH18O2xrPceLiIhIz9VhETezzxO0uPdudQ28Bvi/Ygbrbhkrh9oRcccQERGJrFBLfBbwGPBtIH/EtU3uvrZoqWKQJgt9dTpdRESSo1ARd3d/28wua/2GmQ3qTYW83JsgXR53DBERkciitMRPB54HHHa6kdqBvYuUq9v1y22Gssq4Y4iIiETWYRF399PD595975U7fWiEvjVxJxEREYks6tjpR5lZVbh8kZl938yKM3RaDFLkgoV0n3iDiIiIdELUsdN/DGwxs4OBfwXeAO4uWqpuVkZTsJCOesediIhI/KIW8Yy7O8EsZLe4+60Et5n1CuXeFHcEERGRTova9NxkZv8PuBj4aDh5Sa/pyl3ujcFCLhNvEBERkU6I2hI/H9gO/KO7v0cwmcl3i5aqmxkeLFR1/eQqIiIixRKpiIeF+5dAfzM7Hdjm7r8oarJu1NKxzaL+TSMiIhK/qL3TzwOeAz4BnAfMM7NzixmsO6W8uYin4w0iIiLSCVGviV8NHO7uqwDMbAjwFPDrYgXrTi0t8ZSKuIiIJEfU88ep5gIequ/EZ3s8nU4XEZEkitoSf9zMngB+Fb4+n1ZTjCZZS8c2FXEREUmQSEXc3b9kZv8fcHS46nZ3f6B4sbpXyrPBgoq4iIgkSKH5xCcA3wP2AV4Gvujuy7sjWHfq69uDBRVxERFJkEJV607gEeAcgpnMflj0RDFouSae1chtIiKSHIVOp9e4+x3h8mtm9rdiB4pDmvB0erUGexERkeQoVMQrzOxQdswjXpn/2t17RVEvax47PdVrRpIVEZESUKiIrwS+n/f6vbzXDhxfjFDdrcK3BQtpFXEREUmODou4ux/XXUHiVOFbwyXrcDsREZGeRN2xgVzzYaiojTeIiIhIJ6iIk3efeCrq2DciIiLxUxEnr3e6iriIiCRI1FnMzMwuMrOvha/Hmtnk4kbrPiriIiKSRFFb4j8CjgSmh683AbcWJVEMBmfDuV1UxEVEJEGiVq0p7v5hM3sBwN3XmVmfIubqVlusOlgor4w3iIiISCdEbYk3mVma4N7w5vnEc0VL1c00n7iIiCRR1CJ+M/AAMNTMvgn8GfhW0VJ1o1nzlrJ8bUPwwlTERUQkOaJORfpLM3semEowIspZ7v5qUZN1k4cWLOfwlpa4romLiEhyRKpaZjYW2AI8nL/O3ZcWK1h3GjewDzSg0+kiIpIoUZuevyO4Hm5ABbAX8BrwwSLl6lbBNXED07CrIiKSHFFPpx+Y/9rMPgx8oSiJYhBMgOJxxxAREemU3RqxLZyCdEoXZ4lNdW5T3BFEREQ6Leo18avyXqaADwMripIoBtutAsoq4o4hIiLSKVGvidfkLWcIrpH/puvjxKPcG6FyUNwxREREOqVgEQ8Healx9y92Q55YjMy8C+WaC0ZERJKlw8plZmXungWO6qY8sdiQGgiNDXHHEBER6ZRCLfHnCK5/LzCz2cD9wObmN939t0XM1m0Mh/5j4o4hIiLSKVGviVcA9cDx7Lhf3IFeUsRzukdcREQSp1ARHxr2TH+FHcW7Wa+5sTpFDkzXxEVEJFkKFfE0UM3OxbtZrynihmvIVRERSZxCRXylu9/QLUlilPKcZjATEZHEKXQOuSQuFBuu0+kiIpI4hSrX1G5JETMVcRERSaIOK5e7r+2uIHFSxzYREUkiVS7UEhcRkWRS5QKGZ5brPnEREUkcFXHCYVcbVsUdQ0REpFNUxIE0WajbN+4YIiIinaIiDqQ9A+nyuGOIiIh0ioo4UJPbCOk+cccQERHpFBVxoNK3wraNcccQERHpFBVxoNH6QFVd3DFEREQ6RUWc8D7xvrVxxxAREekUFXEgRVazmImISOKoiANpz0Gq0IRuIiIiPYuKOFBOk4q4iIgkTskX8bQ3BQub18QbREREpJNUxMkGCwPHxRtERESkk0q+iKc8FyyYOraJiEiyqIgTFnH1ThcRkYRREUctcRERSSYVcVdLXEREkqnki7i1tMRL/lCIiEjClHzlSqmIi4hIQpV05Zo1bymL3l0ddwwREZHdUtJF/KEFyylrvk882xRvGBERkU4q6SIOcMiommBBU5GKiEjCFLWIm9nJZvaamS0xs6+08f5VZrbIzF4ys9+bWbcPm6ZbzEREJKmKVsTNLA3cCpwCTASmm9nEVpu9AExy94OAXwPfKVae9miwFxERSapitsQnA0vc/U13bwTuAablb+DuT7v7lvDlX4DRRczTJrXERUQkqYpZxEcB7+a9Xhaua8+ngceKmKdNGuxFRESSqkdMom1mFwGTgGPaef9S4FKAsWPHdum+U82909USFxGRhClmS3w5MCbv9ehw3U7M7ATgauBMd9/e1he5++3uPsndJw0ZMqRLQ5Z5JlhIlXxHfRERSZhiVq6/AhPMbC8z6wNcAMzO38DMDgV+QlDAVxUxS7vKCIt4um8cuxcREdltRSvi7p4BLgeeAF4F7nP3hWZ2g5mdGW72XaAauN/MFpjZ7Ha+rvh0TVxERBKmqNfE3f1R4NFW676Wt3xCMfcfRUvvdCzWHCIiIp2lC8EePmsCFBERSZiSr1yailRERJKq5CtXqrkprrPpIiKSMCVfxFvOp6slLiIiCVPylSu146J4rDlEREQ6q+SLuFriIiKSVCVfuVrGTje1xEVEJFlKvoi3UEtcREQSpuQrV0q3mImISEKVfOUydWwTEZGEUhFXxzYREUmokq9c/XKbgwV1bBMRkYQp+SKeIhsslGkqUhERSZaSL+J9vClYqBgQaw4REZHOKvkiPjy7PFgor4w3iIiISCeVfBFvsJpgIV0ebxAREZFOKvkiniIHaV0PFxGR5Cn5Im7kIJWOO4aIiEinlXwRT5HTPeIiIpJIJV+9Uu5gaomLiEjyqIiT00AvIiKSSCVfxHVNXEREkqrki7iuiYuISFKVfPUq8wyawUxERJKoLO4AcRucXQW+Pe4YIiIinVbyLfFNqf46nS4iIolU8tUrTQaqh8cdQ0REpNNUxD2jcdNFRCSRSr6Il3uTiriIiCRSyRfxat8EfWvjjiEiItJpJV/EyzwD5f3ijiEiItJpJV/EDdewqyIikkglX8RTGnZVREQSSkXcs5rFTEREEklFXC1xERFJKBVxcmqJi4hIIqmIu1riIiKSTCVfxAfnVkMuG3cMERGRTiv5It5g1dC0Oe4YIiIinVbyRRyA2lFxJxAREem0ki/iQce2kj8MIiKSQCVfvVTERUQkqUq+eqVcRVxERJKp5KuXBnsREZGkKvkibhrsRUREEqrki3ha18RFRCShyuIOECfzXLCg0+kikhAbN25k1apVNDU1xR1FulBVVRWjR48mlepco7Kki3ilbwkWyiriDSIiEsHGjRt5//33GTVqFJWVlZhZ3JGkC+RyOZYvX86aNWsYOnRopz5b0ueR04TDrZZXxhtERCSCVatWMWrUKPr166cC3oukUimGDRvGhg0bOv/ZIuRJjJROp4tIgjQ1NVFZqUZHb1ReXk4mk+n050q7iDe3xNU7XUQSQi3w3ml3f68lXsSbW+Il3TVARES6UHV1NW+++Wa37Kuki3jaw5a4TqeLiOyx8ePHU1lZSXV1NcOHD2fGjBk0NDTstM2cOXM4/vjjqampoX///pxxxhksWrRop202btzIzJkzGTt2LNXV1eyzzz7MnDmTNWvWFDX/M888w+jRo/f4exoaGth77727IFFhJV3E+/r2YEGn00VEusTDDz9MQ0MDCxYs4IUXXuDb3/52y3tz587lxBNPZNq0aaxYsYK33nqLgw8+mKOOOqql5drY2MjUqVNZuHAhjz/+OBs3bmTu3LnU1dXx3HPPxfVjtdid69bFVNJFvOV0emZbvEFERHqZ4cOHc9JJJ7FgwYKWdV/+8pe55JJLuPLKK6mpqWHQoEF84xvf4IgjjuD6668H4Be/+AVLly7lgQceYOLEiaRSKYYOHcq1117Lqaee2ua+5syZw+GHH07//v05/PDDmTNnTst7xx57LNdeey1HHXUUNTU1nHjiiW226Ddv3swpp5zCihUrqK6uprq6mhUrVnD99ddz7rnnctFFF1FbW8vPf/5znnvuOY488kgGDBjAiBEjuPzyy2lsbGz5LjNjyZIlAMyYMYPLLruM0047jZqaGqZMmcIbb7zRBUc4UNJFPO3hX1TVnbsvT0REOrZs2TIee+wx9t13XwC2bNnCnDlz+MQnPrHLtueddx5PPvkkAE899RQnn3wy1dXVkfazdu1aTjvtNK644grq6+u56qqrOO2006ivr2/ZZtasWfzsZz9j1apVNDY28r3vfW+X76mqquKxxx5j5MiRNDQ00NDQwMiRIwF46KGHOPfcc1m/fj2f/OQnSafT/Od//idr1qxh7ty5/P73v+dHP/pRuxnvuecerrvuOtatW8e+++7L1VdfHelni6Kke3SlCYt4qjzeICIiu+HrDy9k0YqNRd3HxJG1XHfGByNvf9ZZZ2FmNDQ0cPzxx/P1r38dCIptLpdjxIgRu3xmxIgRLa3j+vp6DjvssMj7+93vfseECRO4+OKLAZg+fTo333wzDz/8MDNmzADgU5/6FPvttx8Q/MEwe/bsyN8PcOSRR3LWWWcBUFlZuVO+8ePH89nPfpZnn32WmTNntvn5s88+m8mTJwPwyU9+kquuuqpT++9ISbfEy5o7tqVL+m8ZEZEu8+CDD7Jp0yaeeeYZFi9e3FKcBw4cSCqVYuXKlbt8ZuXKlQwePBiAurq6Nrdpz4oVKxg3btxO68aNG8fy5ctbXg8fPrxluV+/frt0titkzJgxO71+/fXXOf300xk+fDi1tbV89atf7bDT3Z7uvyMlXb2qPfwLVreYiUgCdaaF3N2OOeYYZsyYwRe/+EUefPBBqqqqOPLII7n//vs57rjjdtr2vvvuY+rUqQCccMIJXHPNNWzevJmqqqqC+xk5ciTvvPPOTuuWLl3KySef3OnM7d2r3Xr95z//eQ499FB+9atfUVNTww9+8AN+/etfd3p/XaGkW+LZ5r9hmkduExGRLjNz5kyefPJJXnzxRQBuvPFG7rrrLm6++WY2bdrEunXruOaaa5g7dy7XXXcdABdffDFjxozhnHPOYfHixeRyOerr6/nWt77Fo48+uss+Tj31VF5//XVmzZpFJpPh3nvvZdGiRZx++umdzjts2DDq6+sLDn+6adMmamtrqa6uZvHixfz4xz/u9L66SkkXcfDgqW9NvDFERHqhIUOGcMkll3DDDTcAcPTRR/PEE0/w29/+lhEjRjBu3DheeOEF/vznPzNhwgQA+vbty1NPPcX+++/Pxz/+cWpra5k8eTJr1qxhypQpu+yjrq6ORx55hJtuuom6ujq+853v8Mgjj7Scnu+M/fffn+nTp7P33nszYMAAVqxY0eZ23/ve95g1axY1NTV85jOf4fzzz+/0vrqKuXtsO98dkyZN8vnz53fJd333v77Pl9Z9HS59FkYe0iXfKSJSLK+++ioHHHBA3DGkSNr7/ZrZ8+4+qa3PqCUOYCV+GEREJJFKunq1zGKmIi4iIglU0tXLWlrimhVIRESSR0Uc1BIXEZFEKunqpSIuIiJJVtLVy5onQEGn00VEJHlKvIg3L5T0YRARkYQq6eplLb3T1RIXEZHkKWoRN7OTzew1M1tiZl9p4/2+ZnZv+P48MxtfzDytpdQ7XUREEqxoRdzM0sCtwCnARGC6mU1stdmngXXuvi/wn8B/FCtP29SxTUSkq4wfP57Kykqqq6sZPnw4M2bM2GXGrjlz5nD88cdTU1ND//79OeOMM1i0aNFO22zcuJGZM2cyduxYqqur2WeffZg5c2aHM4V1hWeeeYbRo0d3yXcde+yx/PSnP+2S7+pIMavXZGCJu7/p7o3APcC0VttMA+4Kl38NTLX2ppEpgpSKuIhIl3r44YdpaGhgwYIFvPDCC3z7299ueW/u3LmceOKJTJs2jRUrVvDWW29x8MEHc9RRR/Hmm28C0NjYyNSpU1m4cCGPP/44GzduZO7cudTV1fHcc8/F9WP1WMWsXqOAd/NeLwvXtbmNu2eADUBdETPtRL3TRUSKY/jw4Zx00kksWLCgZd2Xv/xlLrnkEq688kpqamoYNGgQ3/jGNzjiiCO4/vrrAfjFL37B0qVLeeCBB5g4cSKpVIqhQ4dy7bXXcuqpp7a5rzlz5nD44YfTv39/Dj/8cObMmdPy3rHHHsu1117LUUcdRU1NDSeeeGKbLfrNmzdzyimnsGLFCqqrq6murmbFihXkcjluvPFG9tlnH+rq6jjvvPNYu3YtANu2beOiiy6irq6OAQMGcPjhh/P+++9z9dVX86c//YnLL7+c6upqLr/88q47sK0koglqZpea2Xwzm7969eou+97+g0ewrOqDUNa3y75TRERg2bJlPPbYY+y7774AbNmyhTlz5vCJT3xil23PO+88nnzySQCeeuopTj75ZKqrqyPtZ+3atZx22mlcccUV1NfXc9VVV3HaaadRX1/fss2sWbP42c9+xqpVq2hsbOR73/veLt9TVVXFY489xsiRI2loaKChoYGRI0fywx/+kAcffJBnn32WFStWMHDgQC677DIA7rrrLjZs2MC7775LfX09t912G5WVlXzzm9/kox/9KLfccgsNDQ3ccsstnT5+UZUV7ZthOTAm7/XocF1b2ywzszKgP1Dfahvc/XbgdghmMeuqgNMv/hzwua76OhGR7vXYV+C9l4u7j+EHwik3Rt78rLPOwsxoaGjg+OOP5+tf/zoQFNtcLseIESN2+cyIESNaWsf19fUcdthhkff3u9/9jgkTJnDxxRcDMH36dG6++WYefvhhZsyYAcCnPvUp9ttvPyD4g2H27NmRv/+2227jlltuablWfv311zN27FjuvvtuysvLqa+vZ8mSJRx00EGdyt1VitkS/yswwcz2MrM+wAVA6yM3G/iHcPlc4A+etLlRRUSkxYMPPsimTZt45plnWLx4cUtxHjhwIKlUipUrV+7ymZUrV7bM/11XV9fmNu1ZsWIF48aN22nduHHjWL58R5tx+PDhLcv9+vXbpbNdR9555x3OPvtsBgwYwIABAzjggANIp9O8//77XHzxxZx00klccMEFjBw5ki9/+cs0NTVF/u6uULSWuLtnzOxy4AkgDdzp7gvN7AZgvrvPBv4buNvMlgBrCQq9iIhE0YkWcnc75phjmDFjBl/84hd58MEHqaqq4sgjj+T+++/nuOOO22nb++67j6lTpwJwwgkncM0117B582aqqqoK7mfkyJG88847O61bunQpJ598cqczt9WvesyYMdx5550cddRRbX7muuuu47rrruPtt9/m1FNP5QMf+ACf/vSn2/yuYijqNXF3f9Td93P3fdz9m+G6r4UFHHff5u6fcPd93X2yu79ZzDwiItJ9Zs6cyZNPPsmLL74IwI033shdd93FzTffzKZNm1i3bh3XXHMNc+fO5brrrgPg4osvZsyYMZxzzjksXryYXC5HfX093/rWt3j00Ud32cepp57K66+/zqxZs8hkMtx7770sWrSI008/vdN5hw0bRn19PRs2bGhZ97nPfY6rr7665Q+F1atX89BDDwHw9NNP8/LLL5PNZqmtraW8vJxUKtXyXc097ospER3bREQkeYYMGcIll1zCDTfcAMDRRx/NE088wW9/+1tGjBjBuHHjeOGFF/jzn//MhAkTAOjbty9PPfUU+++/Px//+Mepra1l8uTJrFmzhilTpuyyj7q6Oh555BFuuukm6urq+M53vsMjjzzScnq+M/bff3+mT5/O3nvvzYABA1ixYgVXXnklZ555JieeeCI1NTUcccQRzJs3D4D33nuPc889l9raWg444ACOOeaYlmvzV155Jb/+9a8ZOHAgV1xxxe4ewoIsaZegJ02a5PPnz487hohIt3v11Vc54IAD4o4hRdLe79fMnnf3SW19Ri1xERGRhFIRFxERSSgVcRERkYRSERcREUkoFXERkQRJWmdkiWZ3f68q4iIiCVFeXs7WrVvjjiFF0NTURFlZ58dfUxEXEUmIoUOHsnz5crZs2aIWeS+Sy+V4//336d+/f6c/W8wJUEREpAvV1tYCwXjh3T1GtxRXVVXVbg1QoyIuIpIgtbW1LcVcRKfTRUREEkpFXEREJKFUxEVERBJKRVxERCShEjeLmZmtBt4puGF0g4E1Xfh9pUrHcc/pGO45HcM9p2O457r6GI5z9yFtvZG4It7VzGx+e1O8SXQ6jntOx3DP6RjuOR3DPdedx1Cn00VERBJKRVxERCShVMTh9rgD9BI6jntOx3DP6RjuOR3DPddtx7Dkr4mLiIgklVriIiIiCVUyRdzMTjaz18xsiZl9pY33+5rZveH788xsfAwxe7QIx/AqM1tkZi+Z2e/NbFwcOXuyQscwb7tzzMzNTL2E2xDlOJrZeeG/x4VmNqu7M/Z0Ef57HmtmT5vZC+F/06fGkbOnMrM7zWyVmb3SzvtmZjeHx/clM/twUYK4e69/AGngDWBvoA/wIjCx1TZfAG4Lly8A7o07d096RDyGxwH9wuXP6xh2/hiG29UAfwT+AkyKO3dPe0T8tzgBeAEYGL4eGnfunvSIeAxvBz4fLk8E3o47d096AB8DPgy80s77pwKPAQYcAcwrRo5SaYlPBpa4+5vu3gjcA0xrtc004K5w+dfAVDOzbszY0xU8hu7+tLtvCV/+BRjdzRl7uij/DgH+HfgPYFt3hkuQKMfxM8Ct7r4OwN1XdXPGni7KMXSgebq0/sCKbszX47n7H4G1HWwyDfiFB/4CDDCzEV2do1SK+Cjg3bzXy8J1bW7j7hlgA1DXLemSIcoxzPdpgr9CZYeCxzA85TbG3X/XncESJsq/xf2A/czs/8zsL2Z2crelS4Yox/B64CIzWwY8Cvxz90TrNTr7/8zdovnEpcuZ2UXAJOCYuLMkiZmlgO8DM2KO0huUEZxSP5bgjNAfzexAd18fZ6iEmQ783N1vMrMjgbvN7EPunos7mOxQKi3x5cCYvNejw3VtbmNmZQSnj+q7JV0yRDmGmNkJwNXAme6+vZuyJUWhY1gDfAh4xszeJriONlud23YR5d/iMmC2uze5+1vA6wRFXQJRjuGngfsA3H0uUEEwJrhEE+n/mXuqVIr4X4EJZraXmfUh6Lg2u9U2s4F/CJfPBf7gYe8EASIcQzM7FPgJQQHXNchddXgM3X2Duw929/HuPp6gX8GZ7j4/nrg9VpT/nh8kaIVjZoMJTq+/2Y0Ze7oox3ApMBXAzA4gKOKruzVlss0GLgl7qR8BbHD3lV29k5I4ne7uGTO7HHiCoFfmne6+0MxuAOa7+2zgvwlOFy0h6KxwQXyJe56Ix/C7QDVwf9gncKm7nxlb6B4m4jGUAiIexyeAE81sEZAFvuTuOrMWingM/xW4w8z+haCT2ww1bHYws18R/KE4OOw3cB1QDuDutxH0IzgVWAJsAT5VlBz6nYiIiCRTqZxOFxER6XVUxEVERBJKRVxERCShVMRFREQSSkVcREQkoVTERUREEkpFXBLPzLJmtiDvMb6DbRu6YH8/N7O3wn39LRySsrPf8VMzmxguf7XVe3P2NGP4Pc3H5RUze9jMBhTY/pDdmW7SzEaY2SPh8rFmtiHc76tmdt1ufN+ZzVNjmtlZzccpfH1DOCrgHgl/h+cW2OaZzoyWF/7sj0TYrs0pLM3se2Z2fNT9iYCKuPQOW939kLzH292wzy+5+yHAVwhGqesUd/8nd18Uvvxqq/c+sufxgB3H5UMEAxhdVmD7QwgGp+isq4A78l7/KTw2kwgm0OjUPMruPtvdbwxfnkUwDWbze19z96d2I2NP8nOgrQlZfkjw70kkMhVx6XXMrNrMfh+2kl82s12m+wxbj3/Ma6l+NFx/opnNDT97v5lVF9jdH4F9w89eFX7XK2Y2M1xXZWa/M7MXw/Xnh+ufMbNJZnYjUBnm+GX4XkP4fI+ZnZaX+edmdq6Zpc3su2b2VzN7ycw+G+GwzCWcQcnMJoc/4wtmNsfMPhAOvXkDcH6Y5fww+51m9ly4bVvTpgKcAzzeeqW7bwaeB/YNW/l/CfM+YGYDwyxXmNmicP094boZZnaLmX0EOBP4bphpn7xjcLKZ3Z93bFpawZ39HZrZ18Jj+YqZ3W620xTEF+f9G5kcbh/1uLSpvSks3f0doM7Mhnfm+6TEFXvidD30KPaDYFjNBeHjAYLhhGvD9wYTDHvYPDphQ/j8r8DV4XKaYPKRwQRFuSpc/2/A19rY38+Bc8PlTwDzgMOAl4EqgqFnFwKHEhS4O/I+2z98fgaYlJ8pb5vmjGcDd4XLfQimNawELgWuCdf3BeYDe7WRsyHv57sfODl8XQuUhcsnAL8Jl2cAt+R9/lvAReHyAIJJRKpa7WMv4Pm818cCj4TLdcDbwAeBl4BjwvU3AD8Il1cAfZv30TpH/rHOfx3+jpfm/a5+DFy0m7/DQXnr7wbOyPsd3REufwx4paPj0upnnwT8tIN/s+Obv6/V+juAc+L+b0qP5DxKYux06fW2enD6FgAzKwe+ZWYfA3IELdBhwHt5n/krcGe47YPuvsDMjiE4dft/YWOsD0ELti3fNbNrCCaE+DTBRBEPeND6xMx+C3yUoIV6k5n9B8H/4P/UiZ/rMeC/zKwvwenXP7r7VjM7ETgo75puf4IZut5q9flKM1sQ/vyvAk/mbX+XmU0gGBO7vJ39nwicaWZfDF9XAGPD72o2gl0nxfiomb1AcOxvJJhRbIC7Pxu+fxfBHxUQFPdfmtmDBJOWROLB2N+PA2eY2a+B04AvE0x/G/V32Ow4M/sy0A8YRPAH2MPhe78K9/dHM6u1oF9Be8clP9984J+i/jx5VgEjd+NzUqJUxKU3+iQwBDjM3ZssmNazIn+D8H/KHyP4n//Pzez7wDrgSXefHmEfX3L3Xze/MLOpbW3k7q+H14RPBb5hZr939xui/BDuvs3MngFOAs4H7mneHfDP7v5Ega/Y6u6HmFk/gokuLgNuBv4deNrdz7agE+Az7XzeCFqFr3W0D1odW4Jr4qe3fIlZ/w4+fxpBK/cM4GozO7CDbVu7B7ic4NT0fHffFJ4Kj/o7xMwqgB8RnBV518yuZ+efp/XkEk47x8XMhnUie3sqCI6pSCS6Ji69UX9gVVjAjwPGtd7AzMYB77v7HcBPgQ8TTP15lJk1X+OuMrP9Iu7zT8BZZtbPzKoIToX/ycxGAlvc/X8IZnlrq5NXU3hGoC33Esx+1Nyqh6Agf775M2a2X7jPNrn7FuAK4F/NrIzg+DTPazwjb9NNBJcVmj0B/HPzNWILpppt7XWCU8PtcvcNwDoL+x0AFwPPmlkKGOPuTxOc9u5PcCkiX+tM+Z4lOJ6fYccfOJ39HTYX7DXhtfPWPdab+zAcTTCV5AaiHZfdtR/wSsGtREIq4tIb/RKYZGYvA5cAi9vY5ljgxfC07/nAf7n7aoKi9isze4ngNOz+UXbo7n8juM76HME18p+6+wvAgcBz4Wnt64BvtPHx24GXLOzY1sr/EpwifsrdG8N1PwUWAX+z4Daln1DgrFqY5SVgOvAd4Nvhz57/uaeBic0d2wha7OVhtoXh69bfuxl4o7loduAfCC5BvETQC/4Ggmv1/xP+nl4Abnb39a0+dw/wpbAD2T6t9p0FHgFOCZ/p7O8w3N8dBIXzCYLLLPm2hcfpNoLLJhDhuFjQafGnbe3Tgiks5wIfMLNlZvbpcH05QSdJzR8vkWkqUhHZI2Z2NsGli2vizpJk4XH8sLtfG3cWSQ5dExeRPeLuD5hZXdw5eoEy4Ka4Q0iyqCUuIiKSULomLiIiklAq4iIiIgmlIi4iIpJQKuIiIiIJpSIuIiKSUP8/UaIYmPmqSD8AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "estimate_model(sgd_online)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. 模型比较及结论"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1 三种方案的训练时间、测试时间比较\n",
    "\n",
    "\n",
    "\n",
    "表1：三种方案的时间性能 \n",
    "\n",
    "|  | 逻辑回归模型 | SGD 分类模型 | 在线 SGD 模型 |\n",
    "|:----------: |:----:  | :---: | :----: |\n",
    "| **训练用时** | 10.62s | 6.21s | 1.54s |\n",
    "| **测试用时** | 1.66s  | 1.44s | 0.25s |\n",
    "\n",
    "\n",
    "注：\n",
    "- 以上数据基于 sklearnex 模块对因特尔CPU训练模型加速的结果。\n",
    "- 逻辑回归模型、SGD 分类模型均使用网格搜索的到的最有参数。\n",
    "- 以上时间均取自多次运行的平均用时，不包括数据预处理用时。\n",
    "- 由于主机的硬件等差异，不同主机上各模型的训练时间或有差异。\n",
    "\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2 三种方案的各项指标比较\n",
    "\n",
    "\n",
    "表2：训练集上各项指标\n",
    "\n",
    "|   训练集    | 逻辑回归模型 | SGD 分类模型 | 在线 SGD 模型 |\n",
    "|:---------: | :--------: | :--------: | :----------: |\n",
    "| **准确率**  |  0.95380  |  0.94080   |   0.92600    |\n",
    "| **精度**    |  0.93908  |  0.91563   |   0.89583    |\n",
    "| **召回率**  |  0.97039  |  0.97087   |   0.96138    |\n",
    "| **F1得分**  |  0.95448  |  0.94244   |   0.92745    |\n",
    "| **ROC面积** |  0.98624  |  0.98172   |   0.97636   |\n",
    "\n",
    "\n",
    "<br>\n",
    "\n",
    "\n",
    "表3：测试集上各项指标\n",
    "\n",
    "|   测试集    | 逻辑回归模型 | SGD 分类模型 | 在线 SGD 模型 |\n",
    "|:---------: | :--------: | :--------: | :----------: |\n",
    "| **准确率**  |  0.93280  |  0.93634   |   0.91988    |\n",
    "| **精度**    |  0.91465  |  0.91277   |   0.89059    |\n",
    "| **召回率**  |  0.95428  |  0.96449   |   0.95817    |\n",
    "| **F1得分**  |  0.93404  |  0.93792   |   0.92314    |\n",
    "| **ROC面积** |  0.97764  |  0.97895   |   0.97252    |\n",
    "\n",
    "\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.3 结论\n",
    "\n",
    "#### 模型效率方面：\n",
    "- SGD 分类模型在训练用时均低于逻辑回归超过40%。\n",
    "- 在线学习的 SGD 模型在训练效率和测试效率上大大优于SGD 分类模型(即批量学习的SGD)。\n",
    "- 在线 SGD 模型的训练用时表现极为突出。\n",
    "\n",
    "#### 模型准确率方面：\n",
    "- SGD 分类模型在测试集上大多数指标都略优于逻辑回归，两者不分伯仲。\n",
    "- 在线 SGD 模型在测试机上的指标略低于 批量 SGD 分类模型(即批量学习的SGD)。\n",
    "\n",
    "#### 综上：\n",
    "> **1. SGD 分类模型相比逻辑回归模型，两者在在模型的预测效果上来说不分伯仲，但是SGD 分类模型训练用时、测试用时均占有一定优势。所以在中文文本情感分析上，可优先考虑 SGD 分类模型。**   \n",
    "> \n",
    "> **2. 如若主机的内存有一定限制，或仍对 SGD 分类模型的训练用时不满意，可考虑在线学习，通过小批量的数据投喂，只需要占用极小的内存，即可快速训练出模型。且在训练用时上，在线学习的耗时远低于批量学习。但是在线学习所得模型在预测上的表现可能会低于批量学习。**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. 参考文献\n",
    "\n",
    "`[1]` 李德毅，于剑，中国人工智能学会，中国科协新一代信息技术系列丛书 人工智能导论，中国科学技术出版社，2018.08，第168页        \n",
    "`[2]` 庞宇,基于中文 NLP 的计算机语言学关键问题的研究 [点击链接查看](https://xueshu.baidu.com/usercenter/paper/show?paperid=1q440rb0dd3p0c702f6q0xe06f428290&site=xueshu_se)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10.2 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.2"
  },
  "vscode": {
   "interpreter": {
    "hash": "2c07a9680f6c18190a34a64160181ee75cd3dadc988a7406cfc28b681418c07a"
   }
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
